[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:32:39 dist-test-slave-zpfg krb5kdc[21678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:32:39 dist-test-slave-zpfg krb5kdc[21678](info): set up 2 sockets
Aug 09 00:32:39 dist-test-slave-zpfg krb5kdc[21678](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 00:32:40.900102 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:32:40 dist-test-slave-zpfg krb5kdc[21678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:32:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:32:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:32:41.029443 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35867
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:35867
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:41.112994 21691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:41.113235 21691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:41.113273 21691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:41.114437 21691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:32:41.114500 21691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:41.114524 21691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:41.114553 21691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:32:41.114583 21691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:32:41.116063 21691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:35867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:41.116338 21691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:41.116572 21691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:41.121838 21696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:41.121833 21699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:41.121886 21691 server_base.cc:1034] running on GCE node
W20260809 00:32:41.121833 21697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:41.122466 21691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:41.122767 21691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:41.123932 21691 hybrid_clock.cc:648] HybridClock initialized: now 1786235561123917 us; error 44 us; skew 500 ppm
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235561, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:41.126397 21691 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:32:41.126859 21691 webserver.cc:460] Webserver started at http://127.21.41.254:33387/ using document root <none> and password file <none>
I20260809 00:32:41.127079 21691 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:41.127146 21691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:41.127298 21691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:41.128432 21691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "534000cf3c8a4b4c88e8d713c34e9e64"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "85d1bc4cee88d3f091857bf106914aff"
server_key_iv: "59ac502377510026e28fe16b1ec91858"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.128818 21691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "534000cf3c8a4b4c88e8d713c34e9e64"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "85d1bc4cee88d3f091857bf106914aff"
server_key_iv: "59ac502377510026e28fe16b1ec91858"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.130187 21691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:41.131006 21706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:41.131186 21691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:41.131287 21691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "534000cf3c8a4b4c88e8d713c34e9e64"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "85d1bc4cee88d3f091857bf106914aff"
server_key_iv: "59ac502377510026e28fe16b1ec91858"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.131369 21691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:41.147665 21691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:41.149564 21691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:41.149734 21691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:41.153733 21691 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:35867
I20260809 00:32:41.153789 21758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:35867 every 8 connection(s)
I20260809 00:32:41.154150 21691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:32:41.154663 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 21691
I20260809 00:32:41.154812 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:32:41.155335 21759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:41.155751 21671 external_mini_cluster.cc:1428] Setting key affb9666c4a2f9dabbaf51db2cbb60d5
I20260809 00:32:41.159020 21759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64: Bootstrap starting.
I20260809 00:32:41.159821 21759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:41.160200 21759 log.cc:824] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:41.161001 21759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64: No bootstrap required, opened a new log
I20260809 00:32:41.163058 21759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534000cf3c8a4b4c88e8d713c34e9e64" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35867 } }
I20260809 00:32:41.163228 21759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:41.163260 21759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 534000cf3c8a4b4c88e8d713c34e9e64, State: Initialized, Role: FOLLOWER
I20260809 00:32:41.163370 21759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534000cf3c8a4b4c88e8d713c34e9e64" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35867 } }
I20260809 00:32:41.163437 21759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:32:41.163465 21759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:32:41.163501 21759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:41.164239 21759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534000cf3c8a4b4c88e8d713c34e9e64" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35867 } }
I20260809 00:32:41.164331 21759 leader_election.cc:302] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 534000cf3c8a4b4c88e8d713c34e9e64; no voters:
I20260809 00:32:41.164537 21759 leader_election.cc:288] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:32:41.164580 21764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:41.164726 21764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [term 1 LEADER]: Becoming Leader. State: Replica: 534000cf3c8a4b4c88e8d713c34e9e64, State: Running, Role: LEADER
I20260809 00:32:41.164813 21764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534000cf3c8a4b4c88e8d713c34e9e64" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35867 } }
I20260809 00:32:41.164820 21759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:32:41.166272 21764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 534000cf3c8a4b4c88e8d713c34e9e64. Latest consensus state: current_term: 1 leader_uuid: "534000cf3c8a4b4c88e8d713c34e9e64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534000cf3c8a4b4c88e8d713c34e9e64" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35867 } } }
I20260809 00:32:41.166498 21764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:41.166452 21766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "534000cf3c8a4b4c88e8d713c34e9e64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534000cf3c8a4b4c88e8d713c34e9e64" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35867 } } }
I20260809 00:32:41.166751 21766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64 [sys.catalog]: This master's current role is: LEADER
W20260809 00:32:41.166769 21778 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:32:41.166816 21778 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:32:41.166888 21780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:32:41.167057 21780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:32:41.169360 21780 catalog_manager.cc:1347] Generated new cluster ID: 6ff23c727e4d49219e3780400bd9a4a4
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:41.169411 21780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:32:41.174203 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:41.156807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.21.41.254:35867)
0809 00:32:41.157076 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:41.157083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:41.157215 (+ 132us) server_negotiation.cc:407] Connection header received
0809 00:32:41.160235 (+ 3020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:41.160241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:41.160329 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:41.160440 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:41.162215 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.163694 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.165135 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.165801 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.165928 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:41.170184 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:41.170188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:41.170189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:41.170199 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:41.172103 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.172656 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.172660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.172660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.172771 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.173084 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.173085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.173086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.173380 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:41.173522 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:41.173846 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 00:32:41.173979 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":69,"threads_started":1}
I20260809 00:32:41.180860 21780 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:32:41.181619 21780 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:32:41.184990 21780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 534000cf3c8a4b4c88e8d713c34e9e64: Generated new TSK 0
I20260809 00:32:41.185158 21780 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:32:41.217772 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35867
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:41.307823 21785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:41.308091 21785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:41.308136 21785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:41.309353 21785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:41.309425 21785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:41.309473 21785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:32:41.311070 21785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35867
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:41.311344 21785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:41.311584 21785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:41.316979 21791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:41.316974 21790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:41.316975 21793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:41.317020 21785 server_base.cc:1034] running on GCE node
I20260809 00:32:41.317471 21785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:41.317798 21785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:41.318964 21785 hybrid_clock.cc:648] HybridClock initialized: now 1786235561318951 us; error 33 us; skew 500 ppm
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235561, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:41.321492 21785 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:32:41.321950 21785 webserver.cc:460] Webserver started at http://127.21.41.193:45717/ using document root <none> and password file <none>
I20260809 00:32:41.322192 21785 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:41.322260 21785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:41.322430 21785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:41.323529 21785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "3f49054c93e04a94882c9bfed16dadb9"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "94f28567f23e2ecb28f40676aba08694"
server_key_iv: "01a20abce04cbb20cf4bb3b3e1130cea"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.323918 21785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "3f49054c93e04a94882c9bfed16dadb9"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "94f28567f23e2ecb28f40676aba08694"
server_key_iv: "01a20abce04cbb20cf4bb3b3e1130cea"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.325254 21785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:41.326049 21800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:41.326408 21785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:41.326511 21785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "3f49054c93e04a94882c9bfed16dadb9"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "94f28567f23e2ecb28f40676aba08694"
server_key_iv: "01a20abce04cbb20cf4bb3b3e1130cea"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.326650 21785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:41.358578 21785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:41.360872 21785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:41.361047 21785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:41.361299 21785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:41.361639 21785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:41.361694 21785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:41.361729 21785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:41.361757 21785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:41.368429 21785 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:41269
I20260809 00:32:41.368472 21913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:41269 every 8 connection(s)
I20260809 00:32:41.368829 21785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:32:41.373663 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 21785
I20260809 00:32:41.373775 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:32:41.373952 21671 external_mini_cluster.cc:1428] Setting key bed8af4dd81404e102de2c5c818aacbe
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235561, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:41.379122 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:41.369665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:46669 (local address 127.21.41.254:35867)
0809 00:32:41.369760 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:41.369762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:41.369916 (+ 154us) server_negotiation.cc:407] Connection header received
0809 00:32:41.370435 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:41.370435 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:41.370446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:41.370510 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:41.371500 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.372004 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.373027 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.373251 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.373310 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:41.375618 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:41.375619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:41.375620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:41.375629 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:41.377333 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.377889 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.377890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.377891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.377968 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.378283 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.378284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.378284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.378491 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:41.378555 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:41.378961 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 00:32:41.379051 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 00:32:41.379407 21916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:41.369620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35867 (local address 127.21.41.193:46669)
0809 00:32:41.369786 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:41.369830 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:41.370330 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:41.370566 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:41.370570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:41.371219 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:41.371417 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:41.371422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.372086 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.372087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:41.372937 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:41.372940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.373328 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.373329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:41.373424 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:41.373955 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:41.373963 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:41.375512 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:41.377424 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:41.377433 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:41.377436 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:41.377790 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:41.378050 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:41.378051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:41.378052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:41.378210 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:41.378565 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:41.378573 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:41.378771 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 00:32:41.378908 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 00:32:41.379135 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":98,"threads_started":1}
I20260809 00:32:41.379933 21914 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35867
I20260809 00:32:41.380033 21914 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:41.380246 21914 heartbeater.cc:511] Master 127.21.41.254:35867 requested a full tablet report, sending...
I20260809 00:32:41.380832 21723 ts_manager.cc:194] Registered new tserver with Master: 3f49054c93e04a94882c9bfed16dadb9 (127.21.41.193:41269)
I20260809 00:32:41.381467 21723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:46669
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:32:41.401818 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35867
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:41.491266 21919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:41.491498 21919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:41.491524 21919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:41.492714 21919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:41.492760 21919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:41.492790 21919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:32:41.494501 21919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35867
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:41.494727 21919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:41.494941 21919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:41.500408 21927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:41.500378 21925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:41.500571 21919 server_base.cc:1034] running on GCE node
W20260809 00:32:41.500386 21924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:41.500818 21919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:41.501075 21919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:41.502240 21919 hybrid_clock.cc:648] HybridClock initialized: now 1786235561502222 us; error 37 us; skew 500 ppm
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235561, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:41.504940 21919 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:32:41.505437 21919 webserver.cc:460] Webserver started at http://127.21.41.194:43717/ using document root <none> and password file <none>
I20260809 00:32:41.505667 21919 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:41.505740 21919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:41.505893 21919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:41.507026 21919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "9f2e1c29b78547379a40be7acac80cb4"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "78967a6583a9be25843343c7b692e2fa"
server_key_iv: "a86a0a3fb5c48159c6ca0f5914aef1b5"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.507395 21919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "9f2e1c29b78547379a40be7acac80cb4"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "78967a6583a9be25843343c7b692e2fa"
server_key_iv: "a86a0a3fb5c48159c6ca0f5914aef1b5"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.508854 21919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:32:41.509876 21934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:41.510054 21919 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:41.510159 21919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "9f2e1c29b78547379a40be7acac80cb4"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "78967a6583a9be25843343c7b692e2fa"
server_key_iv: "a86a0a3fb5c48159c6ca0f5914aef1b5"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.510255 21919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:41.517037 21919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:41.519384 21919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:41.519553 21919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:41.519797 21919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:41.520104 21919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:41.520156 21919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:41.520198 21919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:41.520241 21919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:41.526582 21919 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:40325
I20260809 00:32:41.526633 22047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:40325 every 8 connection(s)
I20260809 00:32:41.526943 21919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:32:41.527433 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 21919
I20260809 00:32:41.527503 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:32:41.527706 21671 external_mini_cluster.cc:1428] Setting key 52bc504fa983940fae1969ed9cb8c8d0
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235561, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
I20260809 00:32:41.537935 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:41.527777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:54149 (local address 127.21.41.254:35867)
0809 00:32:41.527834 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:41.527837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:41.528046 (+ 209us) server_negotiation.cc:407] Connection header received
0809 00:32:41.528660 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:41.528662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:41.528674 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:41.528743 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:41.529819 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.530501 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.531759 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.532090 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.532172 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:41.534629 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:41.534630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:41.534631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:41.534639 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:41.536362 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.536841 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.536842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.536843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.536910 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.537209 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.537211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.537211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.537340 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:41.537400 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:41.537743 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 00:32:41.537811 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:32:41.538188 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:41.527752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35867 (local address 127.21.41.194:54149)
0809 00:32:41.527926 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:41.527967 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:41.528566 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:41.528839 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:41.528844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:41.529523 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:41.529727 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:41.529732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.530604 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.530605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:41.531638 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:41.531641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.532189 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.532190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:41.532297 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:41.532935 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:41.532944 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:41.534527 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:41.536455 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:41.536461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:41.536463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:41.536760 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:41.537015 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:41.537016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:41.537016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:41.537134 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:41.537427 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:41.537433 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:41.537575 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 00:32:41.537684 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 00:32:41.537854 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260809 00:32:41.538734 22048 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35867
I20260809 00:32:41.538834 22048 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:41.538980 22048 heartbeater.cc:511] Master 127.21.41.254:35867 requested a full tablet report, sending...
I20260809 00:32:41.539391 21723 ts_manager.cc:194] Registered new tserver with Master: 9f2e1c29b78547379a40be7acac80cb4 (127.21.41.194:40325)
I20260809 00:32:41.539760 21723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:54149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:32:41.558568 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35867
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:41.640992 22053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:41.641232 22053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:41.641270 22053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:41.642494 22053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:41.642552 22053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:41.642598 22053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:32:41.644048 22053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35867
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:41.644320 22053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:41.644547 22053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:41.649878 22061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:41.649885 22059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:41.649885 22053 server_base.cc:1034] running on GCE node
W20260809 00:32:41.650033 22058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:41.650233 22053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:41.650502 22053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:41.651645 22053 hybrid_clock.cc:648] HybridClock initialized: now 1786235561651639 us; error 56 us; skew 500 ppm
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235561, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:41.659346 22053 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:32:41.659817 22053 webserver.cc:460] Webserver started at http://127.21.41.195:44087/ using document root <none> and password file <none>
I20260809 00:32:41.660053 22053 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:41.660120 22053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:41.660274 22053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:41.661347 22053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "7ca31306623e45978ffbc6f29ed87ec1"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "5cea98e85779f8ac2af336214542ff9b"
server_key_iv: "33f148f61f42145051d93466155d306e"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.661713 22053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "7ca31306623e45978ffbc6f29ed87ec1"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "5cea98e85779f8ac2af336214542ff9b"
server_key_iv: "33f148f61f42145051d93466155d306e"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.663133 22053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:32:41.663909 22068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:41.664083 22053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:41.664187 22053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "7ca31306623e45978ffbc6f29ed87ec1"
format_stamp: "Formatted at 2026-08-09 00:32:41 on dist-test-slave-zpfg"
server_key: "5cea98e85779f8ac2af336214542ff9b"
server_key_iv: "33f148f61f42145051d93466155d306e"
server_key_version: "encryptionkey@0"
I20260809 00:32:41.664263 22053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:41.686035 22053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:41.688517 22053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:41.688684 22053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:41.688925 22053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:41.689235 22053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:41.689296 22053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:41.689338 22053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:41.689365 22053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:41.695627 22053 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:38693
I20260809 00:32:41.695708 22181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:38693 every 8 connection(s)
I20260809 00:32:41.696035 22053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235561, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:41.704115 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 22053
I20260809 00:32:41.704211 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:32:41.704389 21671 external_mini_cluster.cc:1428] Setting key 76c0b2c27d53d28600d91c0b6f68d5b1
I20260809 00:32:41.705591 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:41.696823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:45427 (local address 127.21.41.254:35867)
0809 00:32:41.696866 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:41.696868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:41.697082 (+ 214us) server_negotiation.cc:407] Connection header received
0809 00:32:41.697648 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:41.697649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:41.697663 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:41.697713 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:41.698773 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.699289 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.700299 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.700497 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.700582 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:41.702695 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:41.702697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:41.702697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:41.702705 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:41.703849 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.704379 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.704381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.704381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.704464 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.704846 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.704847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.704847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.705002 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:41.705071 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:41.705416 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 00:32:41.705497 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:32:41.705801 22184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:41.696779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35867 (local address 127.21.41.195:45427)
0809 00:32:41.696953 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:41.696996 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:41.697534 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:41.697808 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:41.697818 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:41.698472 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:41.698670 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:41.698675 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.699387 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.699388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:41.700192 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:41.700195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.700593 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.700594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:41.700724 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:41.701181 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:41.701190 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:41.702592 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:41.703977 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:41.703982 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:41.703985 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:41.704307 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:41.704541 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:41.704542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:41.704543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:41.704689 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:41.705070 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:41.705077 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:41.705267 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 00:32:41.705394 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 00:32:41.705555 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260809 00:32:41.706244 22182 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35867
I20260809 00:32:41.706372 22182 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:41.706547 22182 heartbeater.cc:511] Master 127.21.41.254:35867 requested a full tablet report, sending...
I20260809 00:32:41.706907 21723 ts_manager.cc:194] Registered new tserver with Master: 7ca31306623e45978ffbc6f29ed87ec1 (127.21.41.195:38693)
I20260809 00:32:41.707288 21723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:45427
I20260809 00:32:41.716442 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235561, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235561, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.41.193@KRBTEST.COM
I20260809 00:32:41.740211 22189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:41.732360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.21.41.193:41269)
0809 00:32:41.732560 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:41.732570 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:41.732581 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:32:41.732607 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:41.732609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:41.732623 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:41.732690 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:41.733267 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.734055 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.734858 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.735173 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.735228 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:41.737400 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:41.737417 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:41.737417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:41.737424 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:41.739028 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.739396 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.739399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.739400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.739461 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.739654 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.739654 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.739655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.739794 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:41.739961 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:41.740007 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 00:32:41.740088 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":123,"threads_started":1}
W20260809 00:32:41.740569 21828 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39756
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[21678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.193@KRBTEST.COM
I20260809 00:32:41.763409 22189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:41.756591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39762 (local address 127.21.41.193:41269)
0809 00:32:41.756701 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:41.756703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:41.756729 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:32:41.756795 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:41.756796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:41.756808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:41.756850 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:41.757418 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.757913 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.758715 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:41.758929 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:41.758984 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:41.761115 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:41.761116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:41.761116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:41.761122 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:41.762549 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.762819 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.762820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.762820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.762866 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:41.763012 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:41.763012 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:41.763012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:41.763088 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:41.763168 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:41.763251 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:32:41.763308 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 00:32:41.763983 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 21785
I20260809 00:32:41.769948 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 21919
I20260809 00:32:41.774593 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 22053
I20260809 00:32:41.779706 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 21691
2026-08-09T00:32:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1939 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[22196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[22196](info): set up 2 sockets
Aug 09 00:32:41 dist-test-slave-zpfg krb5kdc[22196](info): commencing operation
krb5kdc: starting...
W20260809 00:32:42.835592 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:32:42 dist-test-slave-zpfg krb5kdc[22196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:32:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:32:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:32:42.960743 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:36261
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:36511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:36261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:43.045820 22209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:43.046028 22209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:43.046051 22209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:43.047250 22209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:32:43.047291 22209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:43.047307 22209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:43.047343 22209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:32:43.047374 22209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:32:43.048832 22209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:36511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:36261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:36261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:43.049086 22209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:43.049321 22209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:43.054277 22214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:43.054291 22215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:43.054291 22217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:43.054576 22209 server_base.cc:1034] running on GCE node
I20260809 00:32:43.054824 22209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:43.055074 22209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:43.056229 22209 hybrid_clock.cc:648] HybridClock initialized: now 1786235563056221 us; error 33 us; skew 500 ppm
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:43.058748 22209 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:32:43.059250 22209 webserver.cc:460] Webserver started at http://127.21.41.254:38173/ using document root <none> and password file <none>
I20260809 00:32:43.059458 22209 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:43.059499 22209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:43.059589 22209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:43.060665 22209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "831b984f15a845678b961e860b3ebca1"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "cdaab3707333f8a3a903cc4e3e34b0a1"
server_key_iv: "26335bf4cbe1b968f19d5612e022f067"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.061139 22209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "831b984f15a845678b961e860b3ebca1"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "cdaab3707333f8a3a903cc4e3e34b0a1"
server_key_iv: "26335bf4cbe1b968f19d5612e022f067"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.063182 22209 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:32:43.064239 22224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:43.064459 22209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:43.064553 22209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "831b984f15a845678b961e860b3ebca1"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "cdaab3707333f8a3a903cc4e3e34b0a1"
server_key_iv: "26335bf4cbe1b968f19d5612e022f067"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.064632 22209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:43.089043 22209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:43.091104 22209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:43.091261 22209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:43.095201 22209 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:36261
I20260809 00:32:43.095232 22276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:36261 every 8 connection(s)
I20260809 00:32:43.095609 22209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:32:43.095999 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 22209
I20260809 00:32:43.096081 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:32:43.096261 21671 external_mini_cluster.cc:1428] Setting key e780995a5919d2898329e664141e9a8b
I20260809 00:32:43.096707 22277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:43.100070 22277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1: Bootstrap starting.
I20260809 00:32:43.100728 22277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:43.101074 22277 log.cc:824] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:43.101991 22277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1: No bootstrap required, opened a new log
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:43.104241 22277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "831b984f15a845678b961e860b3ebca1" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36261 } }
I20260809 00:32:43.104441 22277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:43.104487 22277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 831b984f15a845678b961e860b3ebca1, State: Initialized, Role: FOLLOWER
I20260809 00:32:43.104640 22277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "831b984f15a845678b961e860b3ebca1" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36261 } }
I20260809 00:32:43.104735 22277 raft_consensus.cc:388] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:32:43.104789 22277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:32:43.104839 22277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:43.105388 22277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "831b984f15a845678b961e860b3ebca1" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36261 } }
I20260809 00:32:43.105484 22277 leader_election.cc:302] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 831b984f15a845678b961e860b3ebca1; no voters:
I20260809 00:32:43.105674 22277 leader_election.cc:288] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:32:43.105710 22282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:43.105851 22282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [term 1 LEADER]: Becoming Leader. State: Replica: 831b984f15a845678b961e860b3ebca1, State: Running, Role: LEADER
I20260809 00:32:43.105943 22277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:32:43.105930 22282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "831b984f15a845678b961e860b3ebca1" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36261 } }
I20260809 00:32:43.106276 22282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 831b984f15a845678b961e860b3ebca1. Latest consensus state: current_term: 1 leader_uuid: "831b984f15a845678b961e860b3ebca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "831b984f15a845678b961e860b3ebca1" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36261 } } }
I20260809 00:32:43.106405 22282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:43.106290 22284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "831b984f15a845678b961e860b3ebca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "831b984f15a845678b961e860b3ebca1" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36261 } } }
I20260809 00:32:43.106566 22284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1 [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:43.106621 22288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:32:43.107156 22288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:32:43.107452 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.096848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.21.41.254:36261)
0809 00:32:43.097038 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.097048 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.097082 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:32:43.097600 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.097609 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.097674 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.097837 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:43.098457 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.099824 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.100617 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.101263 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.101374 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.103838 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:43.103846 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:43.103846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:43.103863 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:43.105764 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.106155 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.106158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.106158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.106263 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.106500 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.106501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.106501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.106810 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:43.106985 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.107076 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.107230 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260809 00:32:43.108865 22288 catalog_manager.cc:1347] Generated new cluster ID: b1415e4bed6a4145a99b178b7c20d4a7
I20260809 00:32:43.108917 22288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:32:43.123751 22288 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:32:43.124459 22288 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:32:43.128103 22288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 831b984f15a845678b961e860b3ebca1: Generated new TSK 0
I20260809 00:32:43.128280 22288 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:32:43.157948 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36261
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:36511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:43.241293 22303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:43.241499 22303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:43.241523 22303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:43.242775 22303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:43.242816 22303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:43.242844 22303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:32:43.244331 22303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:36511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36261
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:43.244565 22303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:43.244822 22303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:43.249954 22311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:43.250061 22303 server_base.cc:1034] running on GCE node
W20260809 00:32:43.249917 22309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:43.249920 22308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:43.250489 22303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:43.250766 22303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:43.251933 22303 hybrid_clock.cc:648] HybridClock initialized: now 1786235563251928 us; error 36 us; skew 500 ppm
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:43.254467 22303 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:32:43.254954 22303 webserver.cc:460] Webserver started at http://127.21.41.193:36909/ using document root <none> and password file <none>
I20260809 00:32:43.255192 22303 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:43.255261 22303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:43.255396 22303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:43.256449 22303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "34463d8437b24def8846c05bc82c78d6"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "fee43eaca2cce5153876e15b716ea3fe"
server_key_iv: "28fa04448fb959e4ba9c17548228c55b"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.256820 22303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "34463d8437b24def8846c05bc82c78d6"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "fee43eaca2cce5153876e15b716ea3fe"
server_key_iv: "28fa04448fb959e4ba9c17548228c55b"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.258189 22303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:43.258972 22318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:43.259135 22303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:43.259236 22303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "34463d8437b24def8846c05bc82c78d6"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "fee43eaca2cce5153876e15b716ea3fe"
server_key_iv: "28fa04448fb959e4ba9c17548228c55b"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.259325 22303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:43.276307 22303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:43.278427 22303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:43.278561 22303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:43.278743 22303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:43.279075 22303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:43.279110 22303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:43.279134 22303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:43.279178 22303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:43.285015 22303 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:35587
I20260809 00:32:43.285059 22431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:35587 every 8 connection(s)
I20260809 00:32:43.285379 22303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:43.293185 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 22303
I20260809 00:32:43.293293 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:32:43.293479 21671 external_mini_cluster.cc:1428] Setting key d4ce148688e6cf3f125ccb715b4489d4
I20260809 00:32:43.295775 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.286123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:54019 (local address 127.21.41.254:36261)
0809 00:32:43.286168 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.286170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.286415 (+ 245us) server_negotiation.cc:407] Connection header received
0809 00:32:43.287498 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.287499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.287514 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.287571 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:43.288228 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.288735 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.289767 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.289978 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.290086 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.292456 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:43.292458 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:43.292458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:43.292466 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:43.294111 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.294674 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.294675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.294675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.294741 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.295005 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.295006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.295006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.295157 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:43.295225 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.295515 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.295632 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:32:43.295847 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.286085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:36261 (local address 127.21.41.193:54019)
0809 00:32:43.286269 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:43.286307 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:43.287368 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:43.287653 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:43.287657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:43.287932 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:43.288123 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.288127 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.288842 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.288843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.289655 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.289657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.290094 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.290095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.290230 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.290789 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:43.290797 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:43.292356 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:43.294205 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:43.294214 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:43.294217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:43.294576 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:43.294810 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:43.294811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:43.294811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:43.294931 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:43.295235 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:43.295241 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:43.295375 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 00:32:43.295458 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:32:43.295621 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":99,"threads_started":1}
I20260809 00:32:43.296505 22432 heartbeater.cc:347] Connected to a master server at 127.21.41.254:36261
I20260809 00:32:43.296633 22432 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:43.296834 22432 heartbeater.cc:511] Master 127.21.41.254:36261 requested a full tablet report, sending...
I20260809 00:32:43.297380 22241 ts_manager.cc:194] Registered new tserver with Master: 34463d8437b24def8846c05bc82c78d6 (127.21.41.193:35587)
I20260809 00:32:43.298080 22241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:54019
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:32:43.321848 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36261
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:36511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:43.404943 22437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:43.405195 22437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:43.405237 22437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:43.406589 22437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:43.406648 22437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:43.406692 22437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:32:43.408227 22437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:36511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36261
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:43.408504 22437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:43.408730 22437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:43.413757 22442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:43.413735 22443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:43.413740 22445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:43.414496 22437 server_base.cc:1034] running on GCE node
I20260809 00:32:43.414783 22437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:43.415052 22437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:43.416216 22437 hybrid_clock.cc:648] HybridClock initialized: now 1786235563416206 us; error 45 us; skew 500 ppm
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:43.418593 22437 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:32:43.419088 22437 webserver.cc:460] Webserver started at http://127.21.41.194:40723/ using document root <none> and password file <none>
I20260809 00:32:43.419332 22437 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:43.419401 22437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:43.419557 22437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:43.420619 22437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "a953336ce47c43bb89c24cfc631f37e6"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "29c49d0b224e26b66abbdce93bfe5203"
server_key_iv: "ad4d5fb023f7288fb7e2690bcaafc4aa"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.420980 22437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "a953336ce47c43bb89c24cfc631f37e6"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "29c49d0b224e26b66abbdce93bfe5203"
server_key_iv: "ad4d5fb023f7288fb7e2690bcaafc4aa"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.422380 22437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:43.423116 22452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:43.423286 22437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:43.423383 22437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "a953336ce47c43bb89c24cfc631f37e6"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "29c49d0b224e26b66abbdce93bfe5203"
server_key_iv: "ad4d5fb023f7288fb7e2690bcaafc4aa"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.423471 22437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:43.458480 22437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:43.460438 22437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:43.460615 22437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:43.460880 22437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:43.461199 22437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:43.461254 22437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:43.461294 22437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:43.461339 22437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:43.467739 22437 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:36559
I20260809 00:32:43.467803 22565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:36559 every 8 connection(s)
I20260809 00:32:43.468158 22437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:43.477358 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 22437
I20260809 00:32:43.477458 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:32:43.477671 21671 external_mini_cluster.cc:1428] Setting key 03eeb72108640c9c4091f6c311d47829
I20260809 00:32:43.478148 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.469067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:60075 (local address 127.21.41.254:36261)
0809 00:32:43.469194 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.469197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.469489 (+ 292us) server_negotiation.cc:407] Connection header received
0809 00:32:43.470560 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.470561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.470572 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.470624 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:43.471222 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.471717 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.472685 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.472877 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.472927 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.475214 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:43.475215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:43.475216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:43.475232 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:43.476529 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.477003 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.477004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.477005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.477071 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.477438 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.477439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.477440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.477604 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:43.477655 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.477999 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.478082 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 00:32:43.478425 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.469069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:36261 (local address 127.21.41.194:60075)
0809 00:32:43.469334 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:43.469384 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:43.470458 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:43.470683 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:43.470687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:43.470948 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:43.471144 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.471149 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.471810 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.471811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.472606 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.472608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.472954 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.472955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.473073 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.473639 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:43.473647 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:43.475112 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:43.476676 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:43.476683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:43.476684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:43.476924 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:43.477144 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:43.477170 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:43.477171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:43.477338 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:43.477694 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:43.477700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:43.477876 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 00:32:43.477973 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 00:32:43.478158 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":92,"threads_started":1}
I20260809 00:32:43.478917 22566 heartbeater.cc:347] Connected to a master server at 127.21.41.254:36261
I20260809 00:32:43.479024 22566 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:43.479228 22566 heartbeater.cc:511] Master 127.21.41.254:36261 requested a full tablet report, sending...
I20260809 00:32:43.479610 22241 ts_manager.cc:194] Registered new tserver with Master: a953336ce47c43bb89c24cfc631f37e6 (127.21.41.194:36559)
I20260809 00:32:43.480032 22241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:60075
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:32:43.505604 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36261
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:36511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:43.590214 22571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:43.590461 22571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:43.590487 22571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:43.591594 22571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:43.591629 22571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:43.591655 22571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:32:43.593212 22571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:36511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36261
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:43.593408 22571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:43.593587 22571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:43.599576 22576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:43.599678 22579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:43.599575 22577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:43.599761 22571 server_base.cc:1034] running on GCE node
I20260809 00:32:43.600073 22571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:43.600332 22571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:43.601502 22571 hybrid_clock.cc:648] HybridClock initialized: now 1786235563601482 us; error 41 us; skew 500 ppm
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:43.604180 22571 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:32:43.604677 22571 webserver.cc:460] Webserver started at http://127.21.41.195:40651/ using document root <none> and password file <none>
I20260809 00:32:43.604920 22571 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:43.604988 22571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:43.605156 22571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:43.606418 22571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "1eec6f39f5e546388e9b3e8f8c5db704"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "8922e3a2fba68b46e133cf00f3d9cb5d"
server_key_iv: "49946ab90687eab7fa810b19bc1d047d"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.606834 22571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "1eec6f39f5e546388e9b3e8f8c5db704"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "8922e3a2fba68b46e133cf00f3d9cb5d"
server_key_iv: "49946ab90687eab7fa810b19bc1d047d"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.608378 22571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:32:43.609206 22586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:43.609400 22571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:43.609506 22571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "1eec6f39f5e546388e9b3e8f8c5db704"
format_stamp: "Formatted at 2026-08-09 00:32:43 on dist-test-slave-zpfg"
server_key: "8922e3a2fba68b46e133cf00f3d9cb5d"
server_key_iv: "49946ab90687eab7fa810b19bc1d047d"
server_key_version: "encryptionkey@0"
I20260809 00:32:43.609591 22571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:43.628053 22571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:43.630021 22571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:43.630187 22571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:43.630523 22571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:43.630858 22571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:43.630913 22571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:43.630956 22571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:43.630982 22571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:43.637566 22571 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:44909
I20260809 00:32:43.637616 22699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:44909 every 8 connection(s)
I20260809 00:32:43.637928 22571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:32:43.641141 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 22571
I20260809 00:32:43.641239 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:32:43.641450 21671 external_mini_cluster.cc:1428] Setting key a308c988d18ca16ccb19e52ad9f3e177
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:43.647327 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.638671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:60987 (local address 127.21.41.254:36261)
0809 00:32:43.638722 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.638723 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.638950 (+ 227us) server_negotiation.cc:407] Connection header received
0809 00:32:43.639841 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.639842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.639853 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.639916 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:43.640545 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.641174 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.642527 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.642711 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.642777 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.644728 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:43.644729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:43.644730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:43.644736 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:43.646147 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.646562 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.646563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.646564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.646627 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.646834 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.646835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.646835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.646934 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:43.646980 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.647198 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.647262 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:32:43.647569 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.638661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:36261 (local address 127.21.41.195:60987)
0809 00:32:43.638829 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:43.638864 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:43.639745 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:43.639968 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:43.639971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:43.640222 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:43.640444 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.640448 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.641286 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.641287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.642455 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.642457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.642788 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.642789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.642884 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.643325 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:43.643333 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:43.644649 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:43.646219 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:43.646224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:43.646225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:43.646488 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:43.646696 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:43.646697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:43.646698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:43.646792 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:43.646984 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:43.646988 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:43.647096 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 00:32:43.647175 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:32:43.647302 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260809 00:32:43.648015 22700 heartbeater.cc:347] Connected to a master server at 127.21.41.254:36261
I20260809 00:32:43.648105 22700 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:43.648238 22700 heartbeater.cc:511] Master 127.21.41.254:36261 requested a full tablet report, sending...
I20260809 00:32:43.648590 22241 ts_manager.cc:194] Registered new tserver with Master: 1eec6f39f5e546388e9b3e8f8c5db704 (127.21.41.195:44909)
I20260809 00:32:43.648960 22241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:60987
I20260809 00:32:43.653268 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:32:43.661041 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.655057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.21.41.254:36261)
0809 00:32:43.655159 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.655162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.655172 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:32:43.655222 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.655223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.655233 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.655283 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:43.655880 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.656379 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.657096 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.657312 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.657397 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.658733 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:43.658734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:43.658734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:43.658742 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:43.659957 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.660295 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.660296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.660296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.660346 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.660508 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.660509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.660509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.660651 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:43.660727 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.660865 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.660928 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 00:32:43.663741 22241 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:32:43.665767 22241 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:32:43.672624 22713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.668078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:36559 (local address 127.0.0.1:43896)
0809 00:32:43.668462 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:43.668472 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:43.668513 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:43.668718 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:43.668721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:43.668723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:43.668802 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.668804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.670196 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.670197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.671612 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.671615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.672308 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.672309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.672388 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.672402 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:32:43.672468 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 00:32:43.672528 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":10,"thread_start_us":85,"threads_started":1}
I20260809 00:32:43.672684 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.668281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60482 (local address 127.21.41.195:44909)
0809 00:32:43.668508 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.668513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.668524 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:32:43.668567 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.668569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.668586 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.668672 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:43.668986 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.670221 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.671587 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.672341 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.672416 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.672473 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.672545 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.672589 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":91,"threads_started":1}
I20260809 00:32:43.672819 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.668331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43896 (local address 127.21.41.194:36559)
0809 00:32:43.668507 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.668513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.668524 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:32:43.668567 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.668569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.668586 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.668672 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:43.668893 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.670071 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.671734 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.672220 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.672308 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.672369 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.672678 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.672720 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":74,"threads_started":1}
I20260809 00:32:43.673027 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.668299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44220 (local address 127.21.41.193:35587)
0809 00:32:43.668604 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.668609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.668789 (+ 180us) server_negotiation.cc:407] Connection header received
0809 00:32:43.668833 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.668835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.668852 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.668921 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:43.669154 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.670180 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.671700 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.672384 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.672439 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.672486 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.672885 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.672938 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":90,"threads_started":1}
I20260809 00:32:43.672819 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.668091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:35587 (local address 127.0.0.1:44220)
0809 00:32:43.668679 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:43.668688 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:43.668736 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:43.669021 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:43.669022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:43.669024 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:43.669078 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.669079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.670290 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.670290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.671614 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.671615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.672522 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.672523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.672593 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.672598 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:32:43.672649 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:32:43.672691 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":71,"thread_start_us":91,"threads_started":1}
I20260809 00:32:43.672624 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.668078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:44909 (local address 127.0.0.1:60482)
0809 00:32:43.668311 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:43.668356 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:43.668426 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:43.668766 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:43.668767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:43.668770 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:43.668868 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.668870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.670388 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.670389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.671488 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.671490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.672419 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.672420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.672481 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.672484 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:32:43.672534 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 00:32:43.672562 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":59,"threads_started":1}
I20260809 00:32:43.673791 22500 tablet_service.cc:1467] Processing CreateTablet for tablet c462519d044c49c6b0a684115f74b7d4 (DEFAULT_TABLE table=test-table [id=b8230ceb32384477a8a6e7a0738c06be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:43.673803 22634 tablet_service.cc:1467] Processing CreateTablet for tablet c462519d044c49c6b0a684115f74b7d4 (DEFAULT_TABLE table=test-table [id=b8230ceb32384477a8a6e7a0738c06be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:43.674077 22500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c462519d044c49c6b0a684115f74b7d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:43.674124 22634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c462519d044c49c6b0a684115f74b7d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:43.676887 22717 tablet_bootstrap.cc:492] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6: Bootstrap starting.
I20260809 00:32:43.676944 22366 tablet_service.cc:1467] Processing CreateTablet for tablet c462519d044c49c6b0a684115f74b7d4 (DEFAULT_TABLE table=test-table [id=b8230ceb32384477a8a6e7a0738c06be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:43.677313 22366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c462519d044c49c6b0a684115f74b7d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:43.677600 22717 tablet_bootstrap.cc:654] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:43.678197 22717 log.cc:824] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:43.679101 22717 tablet_bootstrap.cc:492] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6: No bootstrap required, opened a new log
I20260809 00:32:43.679180 22717 ts_tablet_manager.cc:1397] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:32:43.680858 22720 tablet_bootstrap.cc:492] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6: Bootstrap starting.
I20260809 00:32:43.681226 22717 raft_consensus.cc:348] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a953336ce47c43bb89c24cfc631f37e6" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36559 } } peers { permanent_uuid: "1eec6f39f5e546388e9b3e8f8c5db704" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44909 } } peers { permanent_uuid: "34463d8437b24def8846c05bc82c78d6" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35587 } }
I20260809 00:32:43.681373 22717 raft_consensus.cc:374] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:43.681396 22717 raft_consensus.cc:729] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a953336ce47c43bb89c24cfc631f37e6, State: Initialized, Role: FOLLOWER
I20260809 00:32:43.681468 22720 tablet_bootstrap.cc:654] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:43.681522 22717 consensus_queue.cc:260] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a953336ce47c43bb89c24cfc631f37e6" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36559 } } peers { permanent_uuid: "1eec6f39f5e546388e9b3e8f8c5db704" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44909 } } peers { permanent_uuid: "34463d8437b24def8846c05bc82c78d6" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35587 } }
I20260809 00:32:43.681813 22717 ts_tablet_manager.cc:1428] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:32:43.681824 22566 heartbeater.cc:503] Master 127.21.41.254:36261 was elected leader, sending a full tablet report...
I20260809 00:32:43.682021 22719 tablet_bootstrap.cc:492] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704: Bootstrap starting.
I20260809 00:32:43.682075 22720 log.cc:824] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:43.682720 22719 tablet_bootstrap.cc:654] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:43.682852 22720 tablet_bootstrap.cc:492] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6: No bootstrap required, opened a new log
I20260809 00:32:43.682926 22720 ts_tablet_manager.cc:1397] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:32:43.683229 22719 log.cc:824] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:43.683876 22719 tablet_bootstrap.cc:492] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704: No bootstrap required, opened a new log
I20260809 00:32:43.683926 22719 ts_tablet_manager.cc:1397] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:32:43.684652 22720 raft_consensus.cc:348] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a953336ce47c43bb89c24cfc631f37e6" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36559 } } peers { permanent_uuid: "1eec6f39f5e546388e9b3e8f8c5db704" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44909 } } peers { permanent_uuid: "34463d8437b24def8846c05bc82c78d6" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35587 } }
I20260809 00:32:43.684816 22720 raft_consensus.cc:374] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:43.684860 22720 raft_consensus.cc:729] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34463d8437b24def8846c05bc82c78d6, State: Initialized, Role: FOLLOWER
I20260809 00:32:43.684989 22720 consensus_queue.cc:260] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a953336ce47c43bb89c24cfc631f37e6" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36559 } } peers { permanent_uuid: "1eec6f39f5e546388e9b3e8f8c5db704" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44909 } } peers { permanent_uuid: "34463d8437b24def8846c05bc82c78d6" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35587 } }
I20260809 00:32:43.685304 22720 ts_tablet_manager.cc:1428] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:32:43.685340 22432 heartbeater.cc:503] Master 127.21.41.254:36261 was elected leader, sending a full tablet report...
I20260809 00:32:43.685532 22719 raft_consensus.cc:348] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a953336ce47c43bb89c24cfc631f37e6" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36559 } } peers { permanent_uuid: "1eec6f39f5e546388e9b3e8f8c5db704" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44909 } } peers { permanent_uuid: "34463d8437b24def8846c05bc82c78d6" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35587 } }
I20260809 00:32:43.685700 22719 raft_consensus.cc:374] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:43.685741 22719 raft_consensus.cc:729] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eec6f39f5e546388e9b3e8f8c5db704, State: Initialized, Role: FOLLOWER
I20260809 00:32:43.685855 22719 consensus_queue.cc:260] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a953336ce47c43bb89c24cfc631f37e6" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36559 } } peers { permanent_uuid: "1eec6f39f5e546388e9b3e8f8c5db704" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44909 } } peers { permanent_uuid: "34463d8437b24def8846c05bc82c78d6" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35587 } }
I20260809 00:32:43.686205 22719 ts_tablet_manager.cc:1428] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:32:43.686239 22700 heartbeater.cc:503] Master 127.21.41.254:36261 was elected leader, sending a full tablet report...
W20260809 00:32:43.718742 22567 tablet.cc:2369] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:32:43.783610 22721 raft_consensus.cc:482] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:32:43.783793 22721 raft_consensus.cc:504] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a953336ce47c43bb89c24cfc631f37e6" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36559 } } peers { permanent_uuid: "1eec6f39f5e546388e9b3e8f8c5db704" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44909 } } peers { permanent_uuid: "34463d8437b24def8846c05bc82c78d6" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35587 } }
I20260809 00:32:43.784209 22721 leader_election.cc:288] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1eec6f39f5e546388e9b3e8f8c5db704 (127.21.41.195:44909), 34463d8437b24def8846c05bc82c78d6 (127.21.41.193:35587)
W20260809 00:32:43.786108 22433 tablet.cc:2369] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:32:43.788722 22726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.784617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:44909 (local address 127.21.41.194:60277)
0809 00:32:43.784793 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:43.784805 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:43.784855 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:43.785024 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:43.785025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:43.785027 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:43.785106 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.785108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.786081 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.786082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.787452 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.787455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.788374 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.788375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.788470 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.788474 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:32:43.788585 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 00:32:43.788631 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":50,"thread_start_us":94,"threads_started":1}
I20260809 00:32:43.788718 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.784844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:60277 (local address 127.21.41.195:44909)
0809 00:32:43.784889 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.784892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.784902 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:32:43.784937 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.784938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.784948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.785004 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:43.785225 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.785978 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.787556 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.788282 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.788365 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.788378 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.788609 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.788651 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:32:43.789238 22654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c462519d044c49c6b0a684115f74b7d4" candidate_uuid: "a953336ce47c43bb89c24cfc631f37e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1eec6f39f5e546388e9b3e8f8c5db704" is_pre_election: true
I20260809 00:32:43.789462 22654 raft_consensus.cc:2454] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a953336ce47c43bb89c24cfc631f37e6 in term 0.
I20260809 00:32:43.789530 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.784400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:35587 (local address 127.21.41.194:50113)
0809 00:32:43.784544 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:43.784584 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:43.784637 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:43.785143 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:43.785144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:43.785145 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:43.785209 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.785210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.786085 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.786085 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.787452 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:43.787455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.789262 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.789263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:43.789332 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.789336 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:32:43.789413 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:32:43.789463 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":157}
I20260809 00:32:43.789738 22453 leader_election.cc:302] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1eec6f39f5e546388e9b3e8f8c5db704, a953336ce47c43bb89c24cfc631f37e6; no voters:
I20260809 00:32:43.789875 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.784915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:50113 (local address 127.21.41.193:35587)
0809 00:32:43.784999 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.785001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.785010 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:32:43.785035 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.785036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.785044 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.785098 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:43.785283 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.786006 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.788419 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.789143 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.789213 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.789221 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.789733 (+ 512us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.789810 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:32:43.789906 22721 raft_consensus.cc:2781] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:32:43.790052 22721 raft_consensus.cc:482] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:32:43.790094 22721 raft_consensus.cc:3037] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:43.790246 22386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c462519d044c49c6b0a684115f74b7d4" candidate_uuid: "a953336ce47c43bb89c24cfc631f37e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34463d8437b24def8846c05bc82c78d6" is_pre_election: true
I20260809 00:32:43.790419 22386 raft_consensus.cc:2454] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a953336ce47c43bb89c24cfc631f37e6 in term 0.
I20260809 00:32:43.791095 22721 raft_consensus.cc:504] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a953336ce47c43bb89c24cfc631f37e6" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36559 } } peers { permanent_uuid: "1eec6f39f5e546388e9b3e8f8c5db704" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44909 } } peers { permanent_uuid: "34463d8437b24def8846c05bc82c78d6" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35587 } }
I20260809 00:32:43.791260 22721 leader_election.cc:288] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [CANDIDATE]: Term 1 election: Requested vote from peers 1eec6f39f5e546388e9b3e8f8c5db704 (127.21.41.195:44909), 34463d8437b24def8846c05bc82c78d6 (127.21.41.193:35587)
I20260809 00:32:43.791410 22386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c462519d044c49c6b0a684115f74b7d4" candidate_uuid: "a953336ce47c43bb89c24cfc631f37e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34463d8437b24def8846c05bc82c78d6"
I20260809 00:32:43.791461 22654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c462519d044c49c6b0a684115f74b7d4" candidate_uuid: "a953336ce47c43bb89c24cfc631f37e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1eec6f39f5e546388e9b3e8f8c5db704"
I20260809 00:32:43.791492 22386 raft_consensus.cc:3037] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:43.791548 22654 raft_consensus.cc:3037] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:43.792397 22386 raft_consensus.cc:2454] T c462519d044c49c6b0a684115f74b7d4 P 34463d8437b24def8846c05bc82c78d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a953336ce47c43bb89c24cfc631f37e6 in term 1.
I20260809 00:32:43.792441 22654 raft_consensus.cc:2454] T c462519d044c49c6b0a684115f74b7d4 P 1eec6f39f5e546388e9b3e8f8c5db704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a953336ce47c43bb89c24cfc631f37e6 in term 1.
I20260809 00:32:43.792574 22453 leader_election.cc:302] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1eec6f39f5e546388e9b3e8f8c5db704, a953336ce47c43bb89c24cfc631f37e6; no voters:
I20260809 00:32:43.792716 22721 raft_consensus.cc:2781] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:43.792908 22721 raft_consensus.cc:686] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [term 1 LEADER]: Becoming Leader. State: Replica: a953336ce47c43bb89c24cfc631f37e6, State: Running, Role: LEADER
I20260809 00:32:43.793018 22721 consensus_queue.cc:237] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a953336ce47c43bb89c24cfc631f37e6" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36559 } } peers { permanent_uuid: "1eec6f39f5e546388e9b3e8f8c5db704" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44909 } } peers { permanent_uuid: "34463d8437b24def8846c05bc82c78d6" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35587 } }
I20260809 00:32:43.793798 22241 catalog_manager.cc:5495] T c462519d044c49c6b0a684115f74b7d4 P a953336ce47c43bb89c24cfc631f37e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a953336ce47c43bb89c24cfc631f37e6 (127.21.41.194). New cstate: current_term: 1 leader_uuid: "a953336ce47c43bb89c24cfc631f37e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a953336ce47c43bb89c24cfc631f37e6" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1eec6f39f5e546388e9b3e8f8c5db704" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34463d8437b24def8846c05bc82c78d6" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35587 } health_report { overall_health: UNKNOWN } } }
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.193@KRBTEST.COM
I20260809 00:32:43.821122 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.813600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.21.41.193:35587)
0809 00:32:43.813668 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.813670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.813716 (+ 46us) server_negotiation.cc:407] Connection header received
0809 00:32:43.813766 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.813767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.813776 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.813825 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:43.814334 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.814865 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.815567 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.815755 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.815806 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.818178 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:43.818189 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:43.818190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:43.818196 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:43.820129 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.820461 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.820464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.820464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.820533 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.820692 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.820692 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.820693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.820831 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:43.820934 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.820989 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.821050 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235563, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235563, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.41.193@KRBTEST.COM
I20260809 00:32:43.844777 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.837777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.21.41.193:35587)
0809 00:32:43.837848 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.837850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.837860 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:32:43.837885 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.837886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.837894 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.837944 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:43.838589 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.839088 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.839829 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.840010 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.840060 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.842381 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:43.842382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:43.842382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:43.842391 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:43.843717 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.844057 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.844058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.844059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.844108 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.844282 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.844283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.844283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.844370 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:43.844439 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.844600 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.844668 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.193@KRBTEST.COM
I20260809 00:32:43.870332 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:43.862879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.21.41.193:35587)
0809 00:32:43.862936 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:43.862939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:43.863051 (+ 112us) server_negotiation.cc:407] Connection header received
0809 00:32:43.863137 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:43.863138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:43.863148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:43.863189 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:43.863784 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.864259 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.864997 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:43.865196 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:43.865272 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:43.867660 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:43.867661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:43.867662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:43.867668 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:43.869082 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.869486 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.869487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.869487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.869536 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:43.869787 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:43.869787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:43.869788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:43.869948 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:43.870059 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:43.870199 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 00:32:43.870261 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 00:32:43.870957 22346 tablet_service.cc:2908] Rejecting scan request for tablet c462519d044c49c6b0a684115f74b7d4: Uninitialized: clean time has not yet been initialized
I20260809 00:32:43.871901 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 22303
I20260809 00:32:43.877833 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 22437
I20260809 00:32:43.882445 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 22571
I20260809 00:32:43.887491 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 22209
2026-08-09T00:32:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2102 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22741](info): set up 2 sockets
Aug 09 00:32:43 dist-test-slave-zpfg krb5kdc[22741](info): commencing operation
krb5kdc: starting...
W20260809 00:32:44.935576 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:32:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:32:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:32:45.057451 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35237
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:35237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:45.140002 22754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:45.140252 22754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:45.140290 22754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:45.141399 22754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:32:45.141469 22754 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 00:32:45.141490 22754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:45.141512 22754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:45.141541 22754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:32:45.141564 22754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:32:45.143021 22754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:35237
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:45.143299 22754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:45.143527 22754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:45.148588 22759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:45.148572 22762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:45.148572 22760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:45.148787 22754 server_base.cc:1034] running on GCE node
I20260809 00:32:45.149093 22754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:45.149384 22754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:45.150530 22754 hybrid_clock.cc:648] HybridClock initialized: now 1786235565150514 us; error 38 us; skew 500 ppm
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:45.156939 22754 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:32:45.157338 22754 webserver.cc:460] Webserver started at http://127.21.41.254:38403/ using document root <none> and password file <none>
I20260809 00:32:45.157519 22754 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:45.157569 22754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:45.157668 22754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:45.158784 22754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "986af12b75c34d2d93a092bf40a608bc"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "bf4977aa0651ea5a737add45612c6f1c"
server_key_iv: "55aec3bdb909d61755b5dcdaf193d77e"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.159094 22754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "986af12b75c34d2d93a092bf40a608bc"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "bf4977aa0651ea5a737add45612c6f1c"
server_key_iv: "55aec3bdb909d61755b5dcdaf193d77e"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.160374 22754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:32:45.161166 22769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:45.161368 22754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:45.161489 22754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "986af12b75c34d2d93a092bf40a608bc"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "bf4977aa0651ea5a737add45612c6f1c"
server_key_iv: "55aec3bdb909d61755b5dcdaf193d77e"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.161651 22754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:45.183238 22754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:45.185282 22754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:45.185467 22754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:45.190236 22754 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:35237
I20260809 00:32:45.190346 22831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:35237 every 8 connection(s)
I20260809 00:32:45.190699 22754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:32:45.191489 22832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:45.192852 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 22754
I20260809 00:32:45.192979 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:32:45.193168 21671 external_mini_cluster.cc:1428] Setting key 95635d802c7bc0705950f76f4b064536
I20260809 00:32:45.194525 22832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc: Bootstrap starting.
I20260809 00:32:45.195406 22832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:45.195797 22832 log.cc:824] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:45.196592 22832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc: No bootstrap required, opened a new log
I20260809 00:32:45.199232 22832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986af12b75c34d2d93a092bf40a608bc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35237 } }
I20260809 00:32:45.199465 22832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:45.199512 22832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 986af12b75c34d2d93a092bf40a608bc, State: Initialized, Role: FOLLOWER
I20260809 00:32:45.199673 22832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986af12b75c34d2d93a092bf40a608bc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35237 } }
I20260809 00:32:45.199770 22832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:32:45.199807 22832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:32:45.199867 22832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:45.200704 22832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986af12b75c34d2d93a092bf40a608bc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35237 } }
I20260809 00:32:45.200804 22832 leader_election.cc:302] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 986af12b75c34d2d93a092bf40a608bc; no voters:
I20260809 00:32:45.201025 22832 leader_election.cc:288] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:32:45.201061 22837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:45.201251 22837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [term 1 LEADER]: Becoming Leader. State: Replica: 986af12b75c34d2d93a092bf40a608bc, State: Running, Role: LEADER
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:45.201336 22832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:32:45.201411 22837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986af12b75c34d2d93a092bf40a608bc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35237 } }
I20260809 00:32:45.201743 22837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 986af12b75c34d2d93a092bf40a608bc. Latest consensus state: current_term: 1 leader_uuid: "986af12b75c34d2d93a092bf40a608bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986af12b75c34d2d93a092bf40a608bc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35237 } } }
I20260809 00:32:45.201848 22837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:45.202147 22844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:32:45.202787 22844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:32:45.201767 22839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "986af12b75c34d2d93a092bf40a608bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986af12b75c34d2d93a092bf40a608bc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35237 } } }
I20260809 00:32:45.203979 22839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:45.205085 22844 catalog_manager.cc:1347] Generated new cluster ID: 82ed3a5290f845e9a1a933adf03e1321
I20260809 00:32:45.205160 22844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:32:45.205619 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.193805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40660 (local address 127.21.41.254:35237)
0809 00:32:45.194006 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.194013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.194041 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:32:45.194545 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.194553 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.194612 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.194781 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:45.195722 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.196880 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.198147 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.198706 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.198799 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.201843 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:45.201848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:45.201849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:45.201863 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:45.203864 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.204226 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.204229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.204229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.204348 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.204545 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.204546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.204546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.204822 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:45.204997 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.205358 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.205480 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
I20260809 00:32:45.213410 22844 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:32:45.214190 22844 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:32:45.220585 22844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc: Generated new TSK 0
I20260809 00:32:45.220768 22844 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:32:45.248752 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35237
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:45.338745 22858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:45.338972 22858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:45.339001 22858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:45.340373 22858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:45.340430 22858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:45.340456 22858 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:32:45.340481 22858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:32:45.342177 22858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.41.254:35237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:45.342522 22858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:45.342813 22858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:45.348774 22864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:45.348807 22858 server_base.cc:1034] running on GCE node
W20260809 00:32:45.348696 22866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:45.348696 22863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:45.349159 22858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:45.349411 22858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:45.350575 22858 hybrid_clock.cc:648] HybridClock initialized: now 1786235565350552 us; error 54 us; skew 500 ppm
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:45.353132 22858 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:32:45.353650 22858 webserver.cc:460] Webserver started at http://127.21.41.193:46335/ using document root <none> and password file <none>
I20260809 00:32:45.353943 22858 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:45.354014 22858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:45.354182 22858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:45.355324 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "cfd0800e8ec34a77bb591c4ef3a9a892"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "c63e3f469919fed8b8401dda44bb2a5f"
server_key_iv: "ac20ed5c8dfed1d5d7199bd4332c9bc9"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.355700 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "cfd0800e8ec34a77bb591c4ef3a9a892"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "c63e3f469919fed8b8401dda44bb2a5f"
server_key_iv: "ac20ed5c8dfed1d5d7199bd4332c9bc9"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.357025 22858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:32:45.357764 22873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:45.357942 22858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:45.358042 22858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "cfd0800e8ec34a77bb591c4ef3a9a892"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "c63e3f469919fed8b8401dda44bb2a5f"
server_key_iv: "ac20ed5c8dfed1d5d7199bd4332c9bc9"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.358130 22858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:45.377265 22858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:45.379200 22858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:45.379379 22858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:45.379917 22858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:45.379981 22858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:45.380029 22858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:45.380061 22858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:45.387475 22858 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:41151
I20260809 00:32:45.387562 22988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:41151 every 8 connection(s)
I20260809 00:32:45.387873 22858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:32:45.390100 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.380505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:54859 (local address 127.21.41.254:35237)
0809 00:32:45.380576 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.380578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.380820 (+ 242us) server_negotiation.cc:407] Connection header received
0809 00:32:45.381592 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.381594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.381606 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.381682 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:45.382713 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.383244 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.384313 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.384575 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.384655 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.387387 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:45.387389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:45.387390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:45.387397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:45.388924 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.389293 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.389293 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.389294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.389344 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.389544 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.389545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.389545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.389649 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:45.389697 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.389954 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.390029 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:32:45.390236 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.380475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35237 (local address 127.21.41.193:54859)
0809 00:32:45.380668 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.380718 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.381474 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.381788 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.381793 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.382202 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:45.382596 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.382603 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.383349 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.383350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.384161 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.384163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.384678 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.384679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.384801 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.385345 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:45.385355 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:45.387272 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:45.388994 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.388999 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.389000 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.389211 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.389399 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.389399 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.389400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.389491 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.389723 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:45.389726 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:45.389839 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.389923 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.390038 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260809 00:32:45.390746 22989 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35237
I20260809 00:32:45.390827 22989 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:45.391014 22989 heartbeater.cc:511] Master 127.21.41.254:35237 requested a full tablet report, sending...
I20260809 00:32:45.391495 22785 ts_manager.cc:194] Registered new tserver with Master: cfd0800e8ec34a77bb591c4ef3a9a892 (127.21.41.193:41151)
I20260809 00:32:45.392280 22785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:54859
I20260809 00:32:45.394600 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 22858
I20260809 00:32:45.394682 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:32:45.394884 21671 external_mini_cluster.cc:1428] Setting key ec14156cb333d4f2926a37f06e910075
I20260809 00:32:45.397509 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.391906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:33925 (local address 127.21.41.254:35237)
0809 00:32:45.391945 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.391946 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.392042 (+ 96us) server_negotiation.cc:407] Connection header received
0809 00:32:45.392067 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.392067 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.392076 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.392137 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:45.392439 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.392970 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.393660 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.393864 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.393947 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.394934 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:45.394935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:45.394935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:45.394942 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:45.396356 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.396724 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.396725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.396725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.396786 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.396981 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.396982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.396982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.397104 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:45.397180 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.397354 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.397421 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:32:45.397527 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.391850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35237 (local address 127.21.41.193:33925)
0809 00:32:45.391967 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.391978 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.392036 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.392140 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.392140 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.392285 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:45.392375 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.392377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.393066 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.393067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.393577 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.393578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.393950 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.393951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.394025 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.394453 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:45.394457 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:45.394845 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:45.396457 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.396458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.396459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.396634 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.396851 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.396852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.396852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.396909 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.397174 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:45.397175 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:45.397232 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.397315 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.397419 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:32:45.419364 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35237
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:45.503208 22998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:45.503412 22998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:45.503438 22998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:45.504663 22998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:45.504706 22998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:45.504727 22998 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:32:45.504745 22998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:32:45.506193 22998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.41.254:35237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:45.506446 22998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:45.506693 22998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:45.511644 23003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:45.511646 23006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:45.511860 22998 server_base.cc:1034] running on GCE node
W20260809 00:32:45.511752 23004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:45.512112 22998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:45.512358 22998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:45.513523 22998 hybrid_clock.cc:648] HybridClock initialized: now 1786235565513498 us; error 35 us; skew 500 ppm
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:45.516047 22998 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:32:45.516512 22998 webserver.cc:460] Webserver started at http://127.21.41.194:35853/ using document root <none> and password file <none>
I20260809 00:32:45.516757 22998 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:45.516825 22998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:45.516970 22998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:45.517984 22998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "c025c77b55c0419f9c75841fd8f170e1"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "749c54cc2e7492231f706dd34e45cb27"
server_key_iv: "18059764bc7696f71e5d0da50a5f57f2"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.518384 22998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "c025c77b55c0419f9c75841fd8f170e1"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "749c54cc2e7492231f706dd34e45cb27"
server_key_iv: "18059764bc7696f71e5d0da50a5f57f2"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.519840 22998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:32:45.520576 23014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:45.520767 22998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:32:45.520857 22998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "c025c77b55c0419f9c75841fd8f170e1"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "749c54cc2e7492231f706dd34e45cb27"
server_key_iv: "18059764bc7696f71e5d0da50a5f57f2"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.520941 22998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:45.528779 22998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:45.530653 22998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:45.530822 22998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:45.531383 22998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:45.531440 22998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:45.531482 22998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:45.531502 22998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:45.539391 22998 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:37377
I20260809 00:32:45.539479 23129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:37377 every 8 connection(s)
I20260809 00:32:45.539815 22998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:32:45.541188 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.531772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:37337 (local address 127.21.41.254:35237)
0809 00:32:45.531841 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.531844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.532099 (+ 255us) server_negotiation.cc:407] Connection header received
0809 00:32:45.532603 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.532604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.532618 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.532679 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:45.533335 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.533866 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.535047 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.535266 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.535353 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.537913 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:45.537915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:45.537916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:45.537925 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:45.539685 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.540205 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.540206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.540206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.540267 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.540521 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.540522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.540523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.540725 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:45.540792 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.541048 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.541123 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:32:45.541298 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.531810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35237 (local address 127.21.41.194:37337)
0809 00:32:45.531967 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.532005 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.532501 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.532756 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.532759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.533049 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:45.533240 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.533244 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.533960 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.533961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.534954 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.534956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.535363 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.535364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.535490 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.536050 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:45.536059 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:45.537813 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:45.539768 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.539772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.539773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.540039 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.540339 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.540339 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.540340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.540434 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.540807 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:45.540812 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:45.540921 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.541005 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.541124 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260809 00:32:45.541806 23130 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35237
I20260809 00:32:45.541899 23130 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:45.542076 23130 heartbeater.cc:511] Master 127.21.41.254:35237 requested a full tablet report, sending...
I20260809 00:32:45.542505 22785 ts_manager.cc:194] Registered new tserver with Master: c025c77b55c0419f9c75841fd8f170e1 (127.21.41.194:37377)
I20260809 00:32:45.542886 22785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:37337
I20260809 00:32:45.544644 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 22998
I20260809 00:32:45.544746 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:32:45.544939 21671 external_mini_cluster.cc:1428] Setting key 5eb67ee6045eb809355a47f9646fe10d
I20260809 00:32:45.548878 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.542876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35237 (local address 127.21.41.194:42841)
0809 00:32:45.543021 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.543033 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.543088 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.543231 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.543232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.543441 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:45.543510 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.543512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.544151 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.544152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.544679 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.544681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.545225 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.545225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.545309 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.545754 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:45.545760 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:45.546034 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:45.547743 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.547744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.547745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.547979 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.548204 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.548205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.548205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.548283 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.548637 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:45.548637 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:45.548672 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.548725 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.548804 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260809 00:32:45.548990 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.542978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:42841 (local address 127.21.41.254:35237)
0809 00:32:45.543019 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.543021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.543075 (+ 54us) server_negotiation.cc:407] Connection header received
0809 00:32:45.543137 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.543137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.543148 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.543200 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:45.543589 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.544047 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.544816 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.545126 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.545204 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.546120 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:45.546121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:45.546121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:45.546129 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:45.547644 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.548079 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.548080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.548080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.548135 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.548356 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.548358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.548358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.548564 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:45.548643 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.548768 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.548858 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:32:45.572824 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35237
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:45.657577 23139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:45.657816 23139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:45.657855 23139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:45.659037 23139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:45.659109 23139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:45.659145 23139 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:32:45.659184 23139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:32:45.660663 23139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.41.254:35237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:45.660926 23139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:45.661217 23139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:45.666304 23147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:45.666479 23139 server_base.cc:1034] running on GCE node
W20260809 00:32:45.666458 23145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:45.666311 23144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:45.666813 23139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:45.667068 23139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:45.668227 23139 hybrid_clock.cc:648] HybridClock initialized: now 1786235565668214 us; error 40 us; skew 500 ppm
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:45.670892 23139 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:32:45.671326 23139 webserver.cc:460] Webserver started at http://127.21.41.195:43641/ using document root <none> and password file <none>
I20260809 00:32:45.671567 23139 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:45.671635 23139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:45.671788 23139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:45.672848 23139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "d88114152d11f2046dd469c3b111358f"
server_key_iv: "15c580ddbc9a8acbdae371adab5d9a48"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.673228 23139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "d88114152d11f2046dd469c3b111358f"
server_key_iv: "15c580ddbc9a8acbdae371adab5d9a48"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.674867 23139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:45.675663 23154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:45.675901 23139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:45.676019 23139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e"
format_stamp: "Formatted at 2026-08-09 00:32:45 on dist-test-slave-zpfg"
server_key: "d88114152d11f2046dd469c3b111358f"
server_key_iv: "15c580ddbc9a8acbdae371adab5d9a48"
server_key_version: "encryptionkey@0"
I20260809 00:32:45.676108 23139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:45.703526 23139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:45.706018 23139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:45.706193 23139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:45.706791 23139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:45.706859 23139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:45.706908 23139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:45.706938 23139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:45.713874 23139 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:35669
I20260809 00:32:45.713976 23269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:35669 every 8 connection(s)
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:45.714373 23139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:32:45.717504 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.707259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:41073 (local address 127.21.41.254:35237)
0809 00:32:45.707328 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.707330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.707532 (+ 202us) server_negotiation.cc:407] Connection header received
0809 00:32:45.708147 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.708149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.708161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.708246 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:45.709124 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.709975 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.711568 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.711908 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.711996 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.714780 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:45.714782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:45.714782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:45.714790 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:45.716267 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.716655 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.716656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.716656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.716710 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.716978 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.716978 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.716978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.717085 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:45.717149 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.717379 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.717439 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:32:45.717721 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.707253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35237 (local address 127.21.41.195:41073)
0809 00:32:45.707417 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.707459 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.708036 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.708329 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.708339 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.708740 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:45.709028 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.709037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.710083 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.710084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.711477 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.711480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.712002 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.712003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.712108 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.712677 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:45.712689 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:45.714663 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:45.716357 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.716363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.716365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.716583 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.716797 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.716798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.716798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.716917 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.717147 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:45.717151 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:45.717265 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.717352 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.717472 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260809 00:32:45.718242 23270 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35237
I20260809 00:32:45.718395 23270 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:45.718478 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 23139
I20260809 00:32:45.718564 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:32:45.718595 23270 heartbeater.cc:511] Master 127.21.41.254:35237 requested a full tablet report, sending...
I20260809 00:32:45.718740 21671 external_mini_cluster.cc:1428] Setting key f2ab3e3f073bd82e47fe43e99b3b1fa5
I20260809 00:32:45.719031 22786 ts_manager.cc:194] Registered new tserver with Master: 6a61c0213e6c41a2bf5f9a1cd6d42b2e (127.21.41.195:35669)
I20260809 00:32:45.719527 22786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:41073
I20260809 00:32:45.719667 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:32:45.726181 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.719836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:44117 (local address 127.21.41.254:35237)
0809 00:32:45.719877 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.719879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.719986 (+ 107us) server_negotiation.cc:407] Connection header received
0809 00:32:45.720054 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.720054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.720064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.720108 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:45.720520 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.721296 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.722192 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.722458 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.722512 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.723415 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:45.723416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:45.723416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:45.723423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:45.724923 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.725283 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.725284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.725284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.725341 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.725631 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.725633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.725633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.725759 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:45.725837 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.725992 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.726089 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:32:45.726197 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.719778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35237 (local address 127.21.41.195:44117)
0809 00:32:45.719936 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.719946 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.719999 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.720159 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.720160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.720351 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:45.720417 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.720418 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.721400 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.721401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.722109 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.722112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.722526 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.722528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.722610 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.723019 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:45.723023 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:45.723329 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:45.725027 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.725028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.725028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.725206 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.725419 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.725420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.725421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.725488 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.725829 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:45.725829 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:45.725876 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.725965 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.726074 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:45.744553 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.737344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40664 (local address 127.21.41.254:35237)
0809 00:32:45.737413 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.737415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.737423 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:32:45.737455 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.737456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.737465 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.737531 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:45.738076 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.738670 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.739412 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.739607 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.739702 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.741954 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:45.741957 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:45.741957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:45.741965 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:45.743333 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.743685 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.743686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.743687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.743772 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.743975 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.743976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.743976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.744093 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:45.744194 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.744368 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.744446 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:32:45.746153 22786 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:32:45.747805 22786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:32:45.754257 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.750092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:37377 (local address 127.0.0.1:38104)
0809 00:32:45.750481 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.750492 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.750558 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.750845 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.750849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.750851 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:45.750926 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.750929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.751929 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.751930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.753130 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.753133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.753981 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.753983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.754046 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.754068 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.754104 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.754159 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":109,"threads_started":1}
I20260809 00:32:45.754402 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.750088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:41151 (local address 127.0.0.1:37076)
0809 00:32:45.750272 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.750326 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.750415 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.750982 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.750983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.750984 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:45.751032 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.751033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.752351 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.752352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.753205 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.753208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.753952 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.753953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.754045 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.754059 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.754142 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.754233 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":2,"thread_start_us":57,"threads_started":1}
I20260809 00:32:45.754658 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.750281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38104 (local address 127.21.41.194:37377)
0809 00:32:45.750627 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.750633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.750644 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:32:45.750688 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.750690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.750708 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.750801 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:45.751068 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.751846 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.753217 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.753882 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.754421 (+ 539us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.754491 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.754515 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.754559 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":215,"threads_started":1}
I20260809 00:32:45.754706 23290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.750326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:35669 (local address 127.0.0.1:58846)
0809 00:32:45.750647 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.750655 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.750688 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.750896 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.750896 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.750899 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:45.750981 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.750983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.752324 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.752325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.753473 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.753476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.754477 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.754478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.754538 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.754541 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.754580 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.754617 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":54,"thread_start_us":98,"threads_started":1}
I20260809 00:32:45.754700 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.750215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.21.41.193:41151)
0809 00:32:45.750819 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.750824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.750834 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:32:45.750871 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.750873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.750889 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.750977 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:45.751118 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.752249 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.753279 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.753868 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.753953 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.754013 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.754171 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.754212 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":90,"threads_started":1}
I20260809 00:32:45.755064 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.750493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58846 (local address 127.21.41.195:35669)
0809 00:32:45.750677 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.750681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.750701 (+ 20us) server_negotiation.cc:407] Connection header received
0809 00:32:45.750753 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.750755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.750771 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.750834 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:45.751068 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.752232 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.753556 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.754272 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.754356 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.754901 (+ 545us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.754927 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.754965 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
I20260809 00:32:45.755455 23064 tablet_service.cc:1467] Processing CreateTablet for tablet 3b92c3b3691442cf987fd06308f75a2f (DEFAULT_TABLE table=test-table [id=876d58008c6e45738d62a698670f7960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:45.755713 23204 tablet_service.cc:1467] Processing CreateTablet for tablet 3b92c3b3691442cf987fd06308f75a2f (DEFAULT_TABLE table=test-table [id=876d58008c6e45738d62a698670f7960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:45.755857 23064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b92c3b3691442cf987fd06308f75a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:45.756058 23204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b92c3b3691442cf987fd06308f75a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:45.758582 23293 tablet_bootstrap.cc:492] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Bootstrap starting.
I20260809 00:32:45.758924 23294 tablet_bootstrap.cc:492] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: Bootstrap starting.
I20260809 00:32:45.758833 22923 tablet_service.cc:1467] Processing CreateTablet for tablet 3b92c3b3691442cf987fd06308f75a2f (DEFAULT_TABLE table=test-table [id=876d58008c6e45738d62a698670f7960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:45.759166 22923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b92c3b3691442cf987fd06308f75a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:45.759552 23294 tablet_bootstrap.cc:654] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:45.759642 23293 tablet_bootstrap.cc:654] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:45.759918 23294 log.cc:824] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:45.759936 23293 log.cc:824] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:45.760751 23294 tablet_bootstrap.cc:492] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: No bootstrap required, opened a new log
I20260809 00:32:45.760828 23294 ts_tablet_manager.cc:1397] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:32:45.760887 23293 tablet_bootstrap.cc:492] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: No bootstrap required, opened a new log
I20260809 00:32:45.760954 23293 ts_tablet_manager.cc:1397] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:32:45.762866 23297 tablet_bootstrap.cc:492] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892: Bootstrap starting.
I20260809 00:32:45.762883 23293 raft_consensus.cc:348] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.763055 23293 raft_consensus.cc:374] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:45.763104 23293 raft_consensus.cc:729] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a61c0213e6c41a2bf5f9a1cd6d42b2e, State: Initialized, Role: FOLLOWER
I20260809 00:32:45.763123 23294 raft_consensus.cc:348] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.763222 23293 consensus_queue.cc:260] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.763285 23294 raft_consensus.cc:374] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:45.763326 23294 raft_consensus.cc:729] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c025c77b55c0419f9c75841fd8f170e1, State: Initialized, Role: FOLLOWER
I20260809 00:32:45.763427 23294 consensus_queue.cc:260] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.763506 23293 ts_tablet_manager.cc:1428] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:32:45.763634 23270 heartbeater.cc:503] Master 127.21.41.254:35237 was elected leader, sending a full tablet report...
I20260809 00:32:45.763682 23294 ts_tablet_manager.cc:1428] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:32:45.763902 23130 heartbeater.cc:503] Master 127.21.41.254:35237 was elected leader, sending a full tablet report...
I20260809 00:32:45.764182 23297 tablet_bootstrap.cc:654] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:45.764619 23297 log.cc:824] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:45.765455 23297 tablet_bootstrap.cc:492] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892: No bootstrap required, opened a new log
I20260809 00:32:45.765509 23297 ts_tablet_manager.cc:1397] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:32:45.767165 23297 raft_consensus.cc:348] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.767290 23297 raft_consensus.cc:374] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:45.767311 23297 raft_consensus.cc:729] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd0800e8ec34a77bb591c4ef3a9a892, State: Initialized, Role: FOLLOWER
I20260809 00:32:45.767448 23297 consensus_queue.cc:260] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.767709 23297 ts_tablet_manager.cc:1428] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:32:45.767757 22989 heartbeater.cc:503] Master 127.21.41.254:35237 was elected leader, sending a full tablet report...
I20260809 00:32:45.775691 23298 raft_consensus.cc:482] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:32:45.775838 23298 raft_consensus.cc:504] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.776216 23298 leader_election.cc:288] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c025c77b55c0419f9c75841fd8f170e1 (127.21.41.194:37377), cfd0800e8ec34a77bb591c4ef3a9a892 (127.21.41.193:41151)
I20260809 00:32:45.779999 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.776483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:43927 (local address 127.21.41.193:41151)
0809 00:32:45.776577 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.776579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.776868 (+ 289us) server_negotiation.cc:407] Connection header received
0809 00:32:45.776894 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.776894 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.776904 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.776954 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:45.777080 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.777749 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.779067 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.779595 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.779676 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.779685 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.779876 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.779923 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 00:32:45.780037 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.776369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:37377 (local address 127.21.41.195:60483)
0809 00:32:45.776437 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.776451 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.776498 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.776698 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.776700 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.776701 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:45.776783 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.776786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.777695 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.777696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.778965 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.778968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.779756 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.779758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.779851 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.779855 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.779913 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.779949 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":3}
I20260809 00:32:45.780042 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.776538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:60483 (local address 127.21.41.194:37377)
0809 00:32:45.776583 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.776584 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.776592 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:32:45.776622 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.776623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.776630 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.776694 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:45.776860 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.777617 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.779052 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.779622 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.779700 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.779709 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.779934 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.779979 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:32:45.780030 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.776412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:41151 (local address 127.21.41.195:43927)
0809 00:32:45.776757 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.776767 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.776828 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.776963 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.776963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.776964 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:45.777016 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.777017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.777814 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.777815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.778965 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.778969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.779692 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.779693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.779795 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.779799 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.779848 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.779910 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260809 00:32:45.780375 23084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b92c3b3691442cf987fd06308f75a2f" candidate_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c025c77b55c0419f9c75841fd8f170e1" is_pre_election: true
I20260809 00:32:45.780524 23084 raft_consensus.cc:2454] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a61c0213e6c41a2bf5f9a1cd6d42b2e in term 0.
I20260809 00:32:45.780498 22943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b92c3b3691442cf987fd06308f75a2f" candidate_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" is_pre_election: true
I20260809 00:32:45.780623 22943 raft_consensus.cc:2454] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a61c0213e6c41a2bf5f9a1cd6d42b2e in term 0.
I20260809 00:32:45.780803 23158 leader_election.cc:302] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a61c0213e6c41a2bf5f9a1cd6d42b2e, c025c77b55c0419f9c75841fd8f170e1; no voters:
I20260809 00:32:45.780965 23298 raft_consensus.cc:2781] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:32:45.781021 23298 raft_consensus.cc:482] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:32:45.781045 23298 raft_consensus.cc:3037] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:45.781744 23298 raft_consensus.cc:504] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.781884 23298 leader_election.cc:288] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [CANDIDATE]: Term 1 election: Requested vote from peers c025c77b55c0419f9c75841fd8f170e1 (127.21.41.194:37377), cfd0800e8ec34a77bb591c4ef3a9a892 (127.21.41.193:41151)
I20260809 00:32:45.782040 23084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b92c3b3691442cf987fd06308f75a2f" candidate_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c025c77b55c0419f9c75841fd8f170e1"
I20260809 00:32:45.782087 22943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b92c3b3691442cf987fd06308f75a2f" candidate_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892"
I20260809 00:32:45.782140 22943 raft_consensus.cc:3037] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:45.782135 23084 raft_consensus.cc:3037] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:45.782858 22943 raft_consensus.cc:2454] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a61c0213e6c41a2bf5f9a1cd6d42b2e in term 1.
I20260809 00:32:45.783111 23084 raft_consensus.cc:2454] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a61c0213e6c41a2bf5f9a1cd6d42b2e in term 1.
I20260809 00:32:45.783413 23157 leader_election.cc:302] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a61c0213e6c41a2bf5f9a1cd6d42b2e, cfd0800e8ec34a77bb591c4ef3a9a892; no voters:
I20260809 00:32:45.783562 23298 raft_consensus.cc:2781] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:45.783706 23298 raft_consensus.cc:686] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 1 LEADER]: Becoming Leader. State: Replica: 6a61c0213e6c41a2bf5f9a1cd6d42b2e, State: Running, Role: LEADER
I20260809 00:32:45.783799 23298 consensus_queue.cc:237] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.784587 22785 catalog_manager.cc:5495] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a61c0213e6c41a2bf5f9a1cd6d42b2e (127.21.41.195). New cstate: current_term: 1 leader_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } health_report { overall_health: HEALTHY } } }
W20260809 00:32:45.790416 23131 tablet.cc:2369] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:32:45.790670 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.788168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.21.41.195:35669)
0809 00:32:45.788229 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.788230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.788253 (+ 23us) server_negotiation.cc:407] Connection header received
0809 00:32:45.788279 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.788280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.788294 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.788344 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:45.788534 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.788986 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.789778 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.789952 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.790002 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.790264 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:45.790396 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:45.790467 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.790543 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.790566 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:32:45.792500 22943 raft_consensus.cc:1261] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892 [term 1 FOLLOWER]: Refusing update from remote peer 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:45.792822 23303 consensus_queue.cc:1035] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:45.794108 23084 raft_consensus.cc:1261] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1 [term 1 FOLLOWER]: Refusing update from remote peer 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:45.794409 23298 consensus_queue.cc:1035] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:45.795648 23306 mvcc.cc:205] Tried to move back new op lower bound from 7316420877483057152 to 7316420877451161600. Current Snapshot: MvccSnapshot[applied={T|T < 7316420877483057152 or (T in {7316420877483057152})}]
I20260809 00:32:45.796028 23307 mvcc.cc:205] Tried to move back new op lower bound from 7316420877483057152 to 7316420877451161600. Current Snapshot: MvccSnapshot[applied={T|T < 7316420877483057152 or (T in {7316420877483057152})}]
I20260809 00:32:45.799034 22786 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:45.799178 22786 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:45.800067 22786 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc: Sending DeleteTablet for 3 replicas of tablet 3b92c3b3691442cf987fd06308f75a2f
I20260809 00:32:45.800450 23064 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b92c3b3691442cf987fd06308f75a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:45 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:38104
I20260809 00:32:45.800462 22923 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b92c3b3691442cf987fd06308f75a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:45 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:37076
I20260809 00:32:45.800510 23204 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b92c3b3691442cf987fd06308f75a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:45 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:58846
I20260809 00:32:45.800824 23316 tablet_replica.cc:330] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: stopping tablet replica
I20260809 00:32:45.800932 23316 raft_consensus.cc:2229] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:32:45.800945 22786 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:32:45.801095 22786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:32:45.801347 23317 tablet_replica.cc:330] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892: stopping tablet replica
I20260809 00:32:45.801462 23317 raft_consensus.cc:2229] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:32:45.801846 23317 raft_consensus.cc:2258] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:45.802050 23316 raft_consensus.cc:2258] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:45.802415 23064 tablet_service.cc:1467] Processing CreateTablet for tablet 9a30a3b692e540e5a107433683f57365 (DEFAULT_TABLE table=test-table [id=88f2adf6f90943e982a88f7b853f6b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:45.802461 22923 tablet_service.cc:1467] Processing CreateTablet for tablet 9a30a3b692e540e5a107433683f57365 (DEFAULT_TABLE table=test-table [id=88f2adf6f90943e982a88f7b853f6b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:45.802551 23064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a30a3b692e540e5a107433683f57365. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:45.802587 22923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a30a3b692e540e5a107433683f57365. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:45.802774 23316 ts_tablet_manager.cc:1905] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:45.803949 23294 tablet_bootstrap.cc:492] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1: Bootstrap starting.
I20260809 00:32:45.804399 23294 tablet_bootstrap.cc:654] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:45.804430 23317 ts_tablet_manager.cc:1905] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:45.804960 23204 tablet_service.cc:1467] Processing CreateTablet for tablet 9a30a3b692e540e5a107433683f57365 (DEFAULT_TABLE table=test-table [id=88f2adf6f90943e982a88f7b853f6b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:45.805076 23204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a30a3b692e540e5a107433683f57365. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:45.805991 23317 ts_tablet_manager.cc:1918] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:32:45.806071 23317 log.cc:1196] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/wal/wals/3b92c3b3691442cf987fd06308f75a2f
I20260809 00:32:45.806357 23317 ts_tablet_manager.cc:1939] T 3b92c3b3691442cf987fd06308f75a2f P cfd0800e8ec34a77bb591c4ef3a9a892: Deleting consensus metadata
I20260809 00:32:45.806726 23293 tablet_bootstrap.cc:492] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Bootstrap starting.
I20260809 00:32:45.806777 22772 catalog_manager.cc:4841] TS cfd0800e8ec34a77bb591c4ef3a9a892 (127.21.41.193:41151): tablet 3b92c3b3691442cf987fd06308f75a2f (table test-table [id=876d58008c6e45738d62a698670f7960]) successfully deleted
I20260809 00:32:45.806954 23294 tablet_bootstrap.cc:492] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1: No bootstrap required, opened a new log
I20260809 00:32:45.806998 23294 ts_tablet_manager.cc:1397] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:32:45.807189 23293 tablet_bootstrap.cc:654] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:45.807215 23294 raft_consensus.cc:348] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.807308 23294 raft_consensus.cc:374] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:45.807340 23294 raft_consensus.cc:729] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c025c77b55c0419f9c75841fd8f170e1, State: Initialized, Role: FOLLOWER
I20260809 00:32:45.807396 23294 consensus_queue.cc:260] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.807497 23294 ts_tablet_manager.cc:1428] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:45.807621 23316 ts_tablet_manager.cc:1918] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:32:45.807690 23316 log.cc:1196] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/3b92c3b3691442cf987fd06308f75a2f
I20260809 00:32:45.807950 23297 tablet_bootstrap.cc:492] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892: Bootstrap starting.
I20260809 00:32:45.808009 23293 tablet_bootstrap.cc:492] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: No bootstrap required, opened a new log
I20260809 00:32:45.808066 23293 ts_tablet_manager.cc:1397] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:32:45.808238 23293 raft_consensus.cc:348] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.808333 23293 raft_consensus.cc:374] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:45.808362 23293 raft_consensus.cc:729] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a61c0213e6c41a2bf5f9a1cd6d42b2e, State: Initialized, Role: FOLLOWER
I20260809 00:32:45.808423 23297 tablet_bootstrap.cc:654] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:45.808422 23293 consensus_queue.cc:260] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.808522 23293 ts_tablet_manager.cc:1428] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:45.808840 23316 ts_tablet_manager.cc:1939] T 3b92c3b3691442cf987fd06308f75a2f P c025c77b55c0419f9c75841fd8f170e1: Deleting consensus metadata
I20260809 00:32:45.809242 22773 catalog_manager.cc:4841] TS c025c77b55c0419f9c75841fd8f170e1 (127.21.41.194:37377): tablet 3b92c3b3691442cf987fd06308f75a2f (table test-table [id=876d58008c6e45738d62a698670f7960]) successfully deleted
I20260809 00:32:45.809319 23318 tablet_replica.cc:330] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: stopping tablet replica
I20260809 00:32:45.809396 23318 raft_consensus.cc:2229] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:32:45.809479 23318 raft_consensus.cc:2258] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:45.809851 23318 ts_tablet_manager.cc:1905] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:45.811256 23318 ts_tablet_manager.cc:1918] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:32:45.811314 23318 log.cc:1196] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/3b92c3b3691442cf987fd06308f75a2f
I20260809 00:32:45.811607 23318 ts_tablet_manager.cc:1939] T 3b92c3b3691442cf987fd06308f75a2f P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Deleting consensus metadata
I20260809 00:32:45.812021 22772 catalog_manager.cc:4841] TS 6a61c0213e6c41a2bf5f9a1cd6d42b2e (127.21.41.195:35669): tablet 3b92c3b3691442cf987fd06308f75a2f (table test-table [id=876d58008c6e45738d62a698670f7960]) successfully deleted
I20260809 00:32:45.815415 23297 tablet_bootstrap.cc:492] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892: No bootstrap required, opened a new log
I20260809 00:32:45.815496 23297 ts_tablet_manager.cc:1397] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260809 00:32:45.815673 23297 raft_consensus.cc:348] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.815765 23297 raft_consensus.cc:374] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:45.815795 23297 raft_consensus.cc:729] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd0800e8ec34a77bb591c4ef3a9a892, State: Initialized, Role: FOLLOWER
I20260809 00:32:45.815869 23297 consensus_queue.cc:260] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.815992 23297 ts_tablet_manager.cc:1428] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 00:32:45.850066 23299 raft_consensus.cc:482] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:32:45.850184 23299 raft_consensus.cc:504] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.850575 23299 leader_election.cc:288] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfd0800e8ec34a77bb591c4ef3a9a892 (127.21.41.193:41151), 6a61c0213e6c41a2bf5f9a1cd6d42b2e (127.21.41.195:35669)
I20260809 00:32:45.853803 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.851068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:50149 (local address 127.21.41.195:35669)
0809 00:32:45.851110 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.851112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.851119 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:32:45.851141 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.851142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.851151 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.851208 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:45.851422 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.852095 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.853045 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.853452 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.853522 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.853532 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.853694 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.853728 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:32:45.853839 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.850903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:35669 (local address 127.21.41.194:50149)
0809 00:32:45.851017 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.851025 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.851059 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.851229 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.851230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.851232 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:45.851334 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.851335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.852171 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.852172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.852951 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.852953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.853544 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.853545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.853614 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.853618 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.853681 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.853720 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260809 00:32:45.854296 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.850798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:41151 (local address 127.21.41.194:34745)
0809 00:32:45.850862 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.850876 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.850929 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.851106 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.851107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.851110 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:45.851186 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.851188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.852140 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.852141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.853338 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.853341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.854054 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.854055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.854140 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.854143 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.854196 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.854232 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":19}
I20260809 00:32:45.854322 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.850881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:34745 (local address 127.21.41.193:41151)
0809 00:32:45.850931 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.850933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.850961 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:32:45.850991 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.850992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.851003 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.851053 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:45.851266 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.852045 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.853443 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.853979 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.854062 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.854070 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.854220 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.854258 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:32:45.854382 23224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a30a3b692e540e5a107433683f57365" candidate_uuid: "c025c77b55c0419f9c75841fd8f170e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" is_pre_election: true
I20260809 00:32:45.854532 23224 raft_consensus.cc:2454] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c025c77b55c0419f9c75841fd8f170e1 in term 0.
I20260809 00:32:45.854581 22943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a30a3b692e540e5a107433683f57365" candidate_uuid: "c025c77b55c0419f9c75841fd8f170e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" is_pre_election: true
I20260809 00:32:45.854679 22943 raft_consensus.cc:2454] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c025c77b55c0419f9c75841fd8f170e1 in term 0.
I20260809 00:32:45.854771 23017 leader_election.cc:302] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a61c0213e6c41a2bf5f9a1cd6d42b2e, c025c77b55c0419f9c75841fd8f170e1; no voters:
I20260809 00:32:45.854921 23299 raft_consensus.cc:2781] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:32:45.854969 23299 raft_consensus.cc:482] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:32:45.854990 23299 raft_consensus.cc:3037] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:45.855894 23299 raft_consensus.cc:504] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.856029 23299 leader_election.cc:288] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [CANDIDATE]: Term 1 election: Requested vote from peers cfd0800e8ec34a77bb591c4ef3a9a892 (127.21.41.193:41151), 6a61c0213e6c41a2bf5f9a1cd6d42b2e (127.21.41.195:35669)
I20260809 00:32:45.856174 22943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a30a3b692e540e5a107433683f57365" candidate_uuid: "c025c77b55c0419f9c75841fd8f170e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892"
I20260809 00:32:45.856205 23224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a30a3b692e540e5a107433683f57365" candidate_uuid: "c025c77b55c0419f9c75841fd8f170e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e"
I20260809 00:32:45.856259 22943 raft_consensus.cc:3037] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:45.856271 23224 raft_consensus.cc:3037] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:45.856997 22943 raft_consensus.cc:2454] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c025c77b55c0419f9c75841fd8f170e1 in term 1.
I20260809 00:32:45.856997 23224 raft_consensus.cc:2454] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c025c77b55c0419f9c75841fd8f170e1 in term 1.
I20260809 00:32:45.857170 23017 leader_election.cc:302] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a61c0213e6c41a2bf5f9a1cd6d42b2e, c025c77b55c0419f9c75841fd8f170e1; no voters:
I20260809 00:32:45.857273 23299 raft_consensus.cc:2781] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:45.857460 23299 raft_consensus.cc:686] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 1 LEADER]: Becoming Leader. State: Replica: c025c77b55c0419f9c75841fd8f170e1, State: Running, Role: LEADER
I20260809 00:32:45.857533 23299 consensus_queue.cc:237] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.858294 22786 catalog_manager.cc:5495] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c025c77b55c0419f9c75841fd8f170e1 (127.21.41.194). New cstate: current_term: 1 leader_uuid: "c025c77b55c0419f9c75841fd8f170e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } health_report { overall_health: UNKNOWN } } }
Aug 09 00:32:45 dist-test-slave-zpfg krb5kdc[22741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
W20260809 00:32:45.888960 22990 tablet.cc:2369] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:32:45.890985 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.883206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40666 (local address 127.21.41.254:35237)
0809 00:32:45.883280 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:45.883283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:45.883403 (+ 120us) server_negotiation.cc:407] Connection header received
0809 00:32:45.883478 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:45.883479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:45.883489 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:45.883540 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:45.884157 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.884637 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.885651 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.885833 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.885878 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.888236 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:45.888237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:45.888238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:45.888248 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:45.889748 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.890203 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.890203 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.890203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.890267 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:45.890503 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:45.890504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:45.890504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:45.890609 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:45.890661 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:45.890849 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 00:32:45.890920 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:32:45.891160 23330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:45.883134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35237 (local address 127.0.0.1:40666)
0809 00:32:45.883314 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:45.883328 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:45.883415 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:45.883573 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:45.883574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:45.884009 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:45.884075 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.884078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.884728 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.884730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.885558 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:45.885561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:45.885910 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:45.885911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:45.885973 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:45.886592 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:45.886598 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:45.888135 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:45.889839 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.889848 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.889850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.890128 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.890334 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:45.890335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:45.890335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:45.890438 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:45.890686 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:45.890693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:45.890747 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 00:32:45.890826 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:32:45.890994 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":45,"thread_start_us":103,"threads_started":1}
I20260809 00:32:45.893169 22786 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:40666:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 00:32:45.893421 22786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:32:45.894877 23204 tablet_service.cc:1467] Processing CreateTablet for tablet d30c7ae2be6d41a4a56175b5e08dd21a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e0a623507f2a41f6973595dd5757f93f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:32:45.894891 22923 tablet_service.cc:1467] Processing CreateTablet for tablet d30c7ae2be6d41a4a56175b5e08dd21a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e0a623507f2a41f6973595dd5757f93f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:32:45.894899 23064 tablet_service.cc:1467] Processing CreateTablet for tablet d30c7ae2be6d41a4a56175b5e08dd21a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e0a623507f2a41f6973595dd5757f93f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:32:45.895035 23204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30c7ae2be6d41a4a56175b5e08dd21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:45.895035 22923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30c7ae2be6d41a4a56175b5e08dd21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:45.895058 23064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30c7ae2be6d41a4a56175b5e08dd21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:45.896575 23293 tablet_bootstrap.cc:492] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Bootstrap starting.
I20260809 00:32:45.896621 23294 tablet_bootstrap.cc:492] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1: Bootstrap starting.
I20260809 00:32:45.897071 23293 tablet_bootstrap.cc:654] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:45.897083 23294 tablet_bootstrap.cc:654] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:45.897396 23297 tablet_bootstrap.cc:492] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: Bootstrap starting.
I20260809 00:32:45.897843 23297 tablet_bootstrap.cc:654] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:45.898465 23293 tablet_bootstrap.cc:492] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: No bootstrap required, opened a new log
I20260809 00:32:45.898481 23297 tablet_bootstrap.cc:492] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: No bootstrap required, opened a new log
I20260809 00:32:45.898530 23293 ts_tablet_manager.cc:1397] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:32:45.898531 23297 ts_tablet_manager.cc:1397] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:32:45.898706 23297 raft_consensus.cc:348] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.898775 23297 raft_consensus.cc:374] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:45.898725 23293 raft_consensus.cc:348] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.898797 23297 raft_consensus.cc:729] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd0800e8ec34a77bb591c4ef3a9a892, State: Initialized, Role: FOLLOWER
I20260809 00:32:45.898814 23293 raft_consensus.cc:374] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:45.898850 23293 raft_consensus.cc:729] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a61c0213e6c41a2bf5f9a1cd6d42b2e, State: Initialized, Role: FOLLOWER
I20260809 00:32:45.898844 23297 consensus_queue.cc:260] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.898902 23293 consensus_queue.cc:260] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.898940 23297 ts_tablet_manager.cc:1428] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 00:32:45.899001 23293 ts_tablet_manager.cc:1428] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:45.899077 23301 tablet_replica.cc:439] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } } }
I20260809 00:32:45.899147 23301 tablet_replica.cc:442] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:45.899250 23313 tablet_replica.cc:439] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } } }
I20260809 00:32:45.899353 23313 tablet_replica.cc:442] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:45.899617 23294 tablet_bootstrap.cc:492] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1: No bootstrap required, opened a new log
I20260809 00:32:45.899672 23294 ts_tablet_manager.cc:1397] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:32:45.899813 23294 raft_consensus.cc:348] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.899874 23294 raft_consensus.cc:374] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:45.899895 23294 raft_consensus.cc:729] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c025c77b55c0419f9c75841fd8f170e1, State: Initialized, Role: FOLLOWER
I20260809 00:32:45.899940 23294 consensus_queue.cc:260] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:45.900029 23294 ts_tablet_manager.cc:1428] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:45.900099 23299 tablet_replica.cc:439] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } } }
I20260809 00:32:45.900174 23299 tablet_replica.cc:442] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 00:32:45.965004 23271 tablet.cc:2369] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:32:46.160841 23301 raft_consensus.cc:482] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:32:46.160961 23301 raft_consensus.cc:504] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:46.161370 23301 leader_election.cc:288] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c025c77b55c0419f9c75841fd8f170e1 (127.21.41.194:37377), 6a61c0213e6c41a2bf5f9a1cd6d42b2e (127.21.41.195:35669)
I20260809 00:32:46.165541 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.161659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:44221 (local address 127.21.41.195:35669)
0809 00:32:46.161707 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:46.161710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:46.161868 (+ 158us) server_negotiation.cc:407] Connection header received
0809 00:32:46.161986 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:46.161987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:46.161998 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:46.162028 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:46.162275 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.163087 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.164647 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.165179 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.165243 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.165251 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:46.165418 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 00:32:46.165470 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:32:46.165591 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.161589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:35669 (local address 127.21.41.193:44221)
0809 00:32:46.161781 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:46.161792 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:46.161900 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:46.162126 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:46.162127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:46.162129 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:46.162193 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:46.162195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.163170 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.163171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:46.164542 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:46.164555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.165255 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.165256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:46.165330 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.165336 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:32:46.165400 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 00:32:46.165451 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":40,"thread_start_us":104,"threads_started":1}
I20260809 00:32:46.165606 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.161911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:33671 (local address 127.21.41.194:37377)
0809 00:32:46.161957 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:46.161959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:46.162069 (+ 110us) server_negotiation.cc:407] Connection header received
0809 00:32:46.162127 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:46.162128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:46.162138 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:46.162226 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:46.162425 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.163153 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.164543 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.165128 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.165221 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.165233 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:46.165476 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 00:32:46.165534 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:32:46.165808 23224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30c7ae2be6d41a4a56175b5e08dd21a" candidate_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" is_pre_election: true
I20260809 00:32:46.165907 23224 raft_consensus.cc:2454] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfd0800e8ec34a77bb591c4ef3a9a892 in term 0.
I20260809 00:32:46.166088 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.161745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:37377 (local address 127.21.41.193:33671)
0809 00:32:46.161983 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:46.161995 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:46.162064 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:46.162249 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:46.162249 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:46.162252 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:46.162331 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:46.162348 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.163221 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.163222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:46.164441 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:46.164444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.165290 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.165291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:46.165364 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.165367 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:32:46.165955 (+ 588us) client_negotiation.cc:240] Negotiation successful
0809 00:32:46.166010 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":27,"thread_start_us":57,"threads_started":1}
I20260809 00:32:46.166085 22876 leader_election.cc:302] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a61c0213e6c41a2bf5f9a1cd6d42b2e, cfd0800e8ec34a77bb591c4ef3a9a892; no voters:
I20260809 00:32:46.166211 23301 raft_consensus.cc:2781] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:32:46.166239 23301 raft_consensus.cc:482] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:32:46.166257 23301 raft_consensus.cc:3037] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:46.166411 23084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30c7ae2be6d41a4a56175b5e08dd21a" candidate_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c025c77b55c0419f9c75841fd8f170e1" is_pre_election: true
I20260809 00:32:46.166504 23084 raft_consensus.cc:2454] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfd0800e8ec34a77bb591c4ef3a9a892 in term 0.
I20260809 00:32:46.167148 23301 raft_consensus.cc:504] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:46.167296 23301 leader_election.cc:288] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [CANDIDATE]: Term 1 election: Requested vote from peers c025c77b55c0419f9c75841fd8f170e1 (127.21.41.194:37377), 6a61c0213e6c41a2bf5f9a1cd6d42b2e (127.21.41.195:35669)
I20260809 00:32:46.167452 23084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30c7ae2be6d41a4a56175b5e08dd21a" candidate_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c025c77b55c0419f9c75841fd8f170e1"
I20260809 00:32:46.167542 23084 raft_consensus.cc:3037] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:46.167554 23224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30c7ae2be6d41a4a56175b5e08dd21a" candidate_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e"
I20260809 00:32:46.167623 23224 raft_consensus.cc:3037] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:46.168282 23224 raft_consensus.cc:2454] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfd0800e8ec34a77bb591c4ef3a9a892 in term 1.
I20260809 00:32:46.168433 22876 leader_election.cc:302] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a61c0213e6c41a2bf5f9a1cd6d42b2e, cfd0800e8ec34a77bb591c4ef3a9a892; no voters:
I20260809 00:32:46.168442 23084 raft_consensus.cc:2454] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfd0800e8ec34a77bb591c4ef3a9a892 in term 1.
I20260809 00:32:46.168504 23301 raft_consensus.cc:2781] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:46.168608 23301 raft_consensus.cc:686] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [term 1 LEADER]: Becoming Leader. State: Replica: cfd0800e8ec34a77bb591c4ef3a9a892, State: Running, Role: LEADER
I20260809 00:32:46.168684 23301 consensus_queue.cc:237] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } }
I20260809 00:32:46.169060 23336 tablet_replica.cc:439] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: TxnStatusTablet state changed. Reason: New leader cfd0800e8ec34a77bb591c4ef3a9a892. Latest consensus state: current_term: 1 leader_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } } }
I20260809 00:32:46.169142 23336 tablet_replica.cc:442] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:32:46.169426 22786 catalog_manager.cc:5495] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 reported cstate change: term changed from 0 to 1, leader changed from <none> to cfd0800e8ec34a77bb591c4ef3a9a892 (127.21.41.193). New cstate: current_term: 1 leader_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } health_report { overall_health: UNKNOWN } } }
I20260809 00:32:46.169834 23338 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 00:32:46.169927 23338 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 00:32:46.187959 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.185169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37080 (local address 127.21.41.193:41151)
0809 00:32:46.185211 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:46.185213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:46.185243 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:32:46.185348 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:46.185349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:46.185366 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:46.185419 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:46.185571 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.186032 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.187185 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.187373 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.187428 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.187641 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:46.187730 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:46.187782 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:46.187857 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 00:32:46.187875 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:32:46.187999 23330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.185116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:41151 (local address 127.0.0.1:37080)
0809 00:32:46.185189 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:46.185208 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:46.185258 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:46.185448 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:46.185449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:46.185451 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:32:46.185506 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:46.185508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.186111 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.186112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:46.187105 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:46.187108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.187450 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.187451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:46.187526 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.187554 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:32:46.187791 (+ 237us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:32:46.187799 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 00:32:46.187855 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:32:46.187913 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 00:32:46.189145 23084 raft_consensus.cc:1261] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1 [term 1 FOLLOWER]: Refusing update from remote peer cfd0800e8ec34a77bb591c4ef3a9a892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:46.189167 23224 raft_consensus.cc:1261] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 1 FOLLOWER]: Refusing update from remote peer cfd0800e8ec34a77bb591c4ef3a9a892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:46.189481 23336 consensus_queue.cc:1035] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:46.189611 23301 consensus_queue.cc:1035] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:46.190249 23313 tablet_replica.cc:439] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: TxnStatusTablet state changed. Reason: New leader cfd0800e8ec34a77bb591c4ef3a9a892. Latest consensus state: current_term: 1 leader_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } } }
I20260809 00:32:46.190373 23313 tablet_replica.cc:442] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:46.190402 23299 tablet_replica.cc:439] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1: TxnStatusTablet state changed. Reason: New leader cfd0800e8ec34a77bb591c4ef3a9a892. Latest consensus state: current_term: 1 leader_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } } }
I20260809 00:32:46.190497 23299 tablet_replica.cc:442] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:46.194486 23336 tablet_replica.cc:439] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } } }
I20260809 00:32:46.194597 23336 tablet_replica.cc:442] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:32:46.194573 23311 mvcc.cc:205] Tried to move back new op lower bound from 7316420879108648960 to 7316420879027650560. Current Snapshot: MvccSnapshot[applied={T|T < 7316420879108648960}]
I20260809 00:32:46.194674 23299 tablet_replica.cc:439] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } } }
I20260809 00:32:46.194773 23299 tablet_replica.cc:442] T d30c7ae2be6d41a4a56175b5e08dd21a P c025c77b55c0419f9c75841fd8f170e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:46.198868 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.196065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.21.41.194:37377)
0809 00:32:46.196102 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:46.196104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:46.196113 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:32:46.196192 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:46.196193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:46.196210 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:46.196261 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:46.196388 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.196863 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.198030 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.198209 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.198262 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.198453 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:46.198583 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:46.198663 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:46.198721 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 00:32:46.198744 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:32:46.198999 23301 tablet_replica.cc:439] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } } }
I20260809 00:32:46.199128 23301 tablet_replica.cc:442] T d30c7ae2be6d41a4a56175b5e08dd21a P cfd0800e8ec34a77bb591c4ef3a9a892: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:32:46.199344 23313 tablet_replica.cc:439] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 } } peers { permanent_uuid: "c025c77b55c0419f9c75841fd8f170e1" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37377 } } peers { permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 } } }
I20260809 00:32:46.199458 23313 tablet_replica.cc:442] T d30c7ae2be6d41a4a56175b5e08dd21a P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:46.204219 23345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.201428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:41151 (local address 127.21.41.194:52043)
0809 00:32:46.201620 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:46.201633 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:46.201683 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:46.201823 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:46.201824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:46.201825 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:32:46.201874 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:46.201875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.202780 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.202781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:46.203349 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:46.203351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.203848 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.203849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:46.203909 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.203921 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:32:46.204094 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:32:46.204095 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:32:46.204127 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 00:32:46.204160 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":126,"threads_started":1}
I20260809 00:32:46.204222 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.201633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:52043 (local address 127.21.41.193:41151)
0809 00:32:46.201698 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:46.201700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:46.201709 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:32:46.201732 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:46.201732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:46.201742 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:46.201810 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:46.201920 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.202700 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.203431 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.203749 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.203825 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.203999 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:46.204032 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:46.204091 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:46.204130 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 00:32:46.204151 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:32:46.206715 22943 raft_consensus.cc:1261] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892 [term 1 FOLLOWER]: Refusing update from remote peer c025c77b55c0419f9c75841fd8f170e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:46.206713 23224 raft_consensus.cc:1261] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 1 FOLLOWER]: Refusing update from remote peer c025c77b55c0419f9c75841fd8f170e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:46.207024 23299 consensus_queue.cc:1035] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd0800e8ec34a77bb591c4ef3a9a892" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:46.207175 23323 consensus_queue.cc:1035] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a61c0213e6c41a2bf5f9a1cd6d42b2e" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:46.216663 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.213819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:37377 (local address 127.21.41.193:43589)
0809 00:32:46.213989 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:46.214001 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:46.214061 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:46.214175 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:46.214176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:46.214189 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:32:46.214250 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:46.214251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.214909 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.214910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:46.215708 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:46.215711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.216189 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.216190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:46.216260 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.216278 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:32:46.216520 (+ 242us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:32:46.216522 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:32:46.216558 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 00:32:46.216593 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260809 00:32:46.216656 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.213845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:43589 (local address 127.21.41.194:37377)
0809 00:32:46.213984 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:46.213986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:46.214048 (+ 62us) server_negotiation.cc:407] Connection header received
0809 00:32:46.214122 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:46.214123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:46.214132 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:46.214175 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:46.214350 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.214822 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.215778 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.216092 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.216173 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.216426 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:46.216460 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:46.216512 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:46.216562 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 00:32:46.216582 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260809 00:32:46.227460 22786 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:46.227578 22786 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:46.228271 22786 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 986af12b75c34d2d93a092bf40a608bc: Sending DeleteTablet for 3 replicas of tablet 9a30a3b692e540e5a107433683f57365
I20260809 00:32:46.228529 22923 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a30a3b692e540e5a107433683f57365 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:46 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:37076
I20260809 00:32:46.228626 23317 tablet_replica.cc:330] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892: stopping tablet replica
I20260809 00:32:46.228605 23204 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a30a3b692e540e5a107433683f57365 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:46 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:58846
I20260809 00:32:46.228668 23317 raft_consensus.cc:2229] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:32:46.228688 23318 tablet_replica.cc:330] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: stopping tablet replica
I20260809 00:32:46.228705 23317 raft_consensus.cc:2258] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:46.228735 23318 raft_consensus.cc:2229] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:32:46.228775 23318 raft_consensus.cc:2258] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:46.228968 23064 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a30a3b692e540e5a107433683f57365 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:46 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:38104
I20260809 00:32:46.229082 23316 tablet_replica.cc:330] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1: stopping tablet replica
I20260809 00:32:46.229131 23316 raft_consensus.cc:2229] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:32:46.229220 23316 raft_consensus.cc:2258] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:46.229573 23318 ts_tablet_manager.cc:1905] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:46.229588 23316 ts_tablet_manager.cc:1905] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:46.231247 23316 ts_tablet_manager.cc:1918] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:32:46.231328 23316 log.cc:1196] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/9a30a3b692e540e5a107433683f57365
I20260809 00:32:46.231547 23316 ts_tablet_manager.cc:1939] T 9a30a3b692e540e5a107433683f57365 P c025c77b55c0419f9c75841fd8f170e1: Deleting consensus metadata
I20260809 00:32:46.231971 22773 catalog_manager.cc:4841] TS c025c77b55c0419f9c75841fd8f170e1 (127.21.41.194:37377): tablet 9a30a3b692e540e5a107433683f57365 (table test-table [id=88f2adf6f90943e982a88f7b853f6b8a]) successfully deleted
I20260809 00:32:46.232147 23318 ts_tablet_manager.cc:1918] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:32:46.232218 23318 log.cc:1196] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/9a30a3b692e540e5a107433683f57365
I20260809 00:32:46.232432 23318 ts_tablet_manager.cc:1939] T 9a30a3b692e540e5a107433683f57365 P 6a61c0213e6c41a2bf5f9a1cd6d42b2e: Deleting consensus metadata
I20260809 00:32:46.232780 22772 catalog_manager.cc:4841] TS 6a61c0213e6c41a2bf5f9a1cd6d42b2e (127.21.41.195:35669): tablet 9a30a3b692e540e5a107433683f57365 (table test-table [id=88f2adf6f90943e982a88f7b853f6b8a]) successfully deleted
I20260809 00:32:46.232887 23317 ts_tablet_manager.cc:1905] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:46.234540 23317 ts_tablet_manager.cc:1918] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:32:46.234611 23317 log.cc:1196] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786235559853195-21671-0/minicluster-data/ts-0/wal/wals/9a30a3b692e540e5a107433683f57365
I20260809 00:32:46.234822 23317 ts_tablet_manager.cc:1939] T 9a30a3b692e540e5a107433683f57365 P cfd0800e8ec34a77bb591c4ef3a9a892: Deleting consensus metadata
I20260809 00:32:46.235175 22772 catalog_manager.cc:4841] TS cfd0800e8ec34a77bb591c4ef3a9a892 (127.21.41.193:41151): tablet 9a30a3b692e540e5a107433683f57365 (table test-table [id=88f2adf6f90943e982a88f7b853f6b8a]) successfully deleted
Aug 09 00:32:46 dist-test-slave-zpfg krb5kdc[22741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.41.193@KRBTEST.COM
I20260809 00:32:46.253014 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.232007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37084 (local address 127.21.41.193:41151)
0809 00:32:46.232057 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:46.232059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:46.234596 (+ 2537us) server_negotiation.cc:407] Connection header received
0809 00:32:46.234799 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:46.234800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:46.234811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:46.234843 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:46.235734 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.236489 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.237657 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.237951 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.238030 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.241988 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:46.241998 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:46.241998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:46.242009 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:46.251548 (+ 9539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:46.252039 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:46.252043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:46.252044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:46.252117 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:46.252351 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:46.252353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:46.252353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:46.252538 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:46.252689 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:46.252821 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 00:32:46.252910 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 00:32:46.253314 22983 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37084
I20260809 00:32:46.262473 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:46.254474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.21.41.254:35237)
0809 00:32:46.254517 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:46.254519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:46.254528 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:32:46.254581 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:46.254582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:46.254594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:46.254654 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:46.255349 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.256166 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.257190 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:46.257511 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:46.257584 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:46.259209 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:46.259210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:46.259210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:46.259218 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:46.261003 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:46.261390 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:46.261391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:46.261392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:46.261474 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:46.261690 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:46.261690 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:46.261691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:46.261865 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:46.261971 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:46.262105 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 00:32:46.262194 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:32:46.262684 22786 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40668
I20260809 00:32:46.263764 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 22858
I20260809 00:32:46.271404 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 22998
I20260809 00:32:46.276549 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 23139
I20260809 00:32:46.282313 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 22754
2026-08-09T00:32:46Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2394 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:32:46 dist-test-slave-zpfg krb5kdc[23359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:32:46 dist-test-slave-zpfg krb5kdc[23359](info): set up 2 sockets
Aug 09 00:32:46 dist-test-slave-zpfg krb5kdc[23359](info): commencing operation
krb5kdc: starting...
W20260809 00:32:47.327682 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:32:47 dist-test-slave-zpfg krb5kdc[23359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:32:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:32:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:32:47.451303 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:33743
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:33743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:47.534631 23372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:47.534852 23372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:47.534873 23372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:47.536062 23372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:32:47.536106 23372 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 00:32:47.536120 23372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:47.536134 23372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:47.536145 23372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:32:47.536157 23372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:32:47.537664 23372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:33743
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:33743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:47.537925 23372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:47.538172 23372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:47.543012 23377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:47.543013 23380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:47.543157 23378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:47.543426 23372 server_base.cc:1034] running on GCE node
I20260809 00:32:47.543620 23372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:47.543870 23372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:47.545009 23372 hybrid_clock.cc:648] HybridClock initialized: now 1786235567544986 us; error 41 us; skew 500 ppm
Aug 09 00:32:47 dist-test-slave-zpfg krb5kdc[23359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235567, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:47.547734 23372 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:32:47.548205 23372 webserver.cc:460] Webserver started at http://127.21.41.254:34789/ using document root <none> and password file <none>
I20260809 00:32:47.548440 23372 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:47.548506 23372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:47.548647 23372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:47.549701 23372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "ee0b8e4020d44b59938f8ab4f47282c5"
format_stamp: "Formatted at 2026-08-09 00:32:47 on dist-test-slave-zpfg"
server_key: "d8bf8376575497f7e8d3a6a82bfbe9f2"
server_key_iv: "3064615d284ff2df4034f59cc9f5081f"
server_key_version: "encryptionkey@0"
I20260809 00:32:47.550058 23372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "ee0b8e4020d44b59938f8ab4f47282c5"
format_stamp: "Formatted at 2026-08-09 00:32:47 on dist-test-slave-zpfg"
server_key: "d8bf8376575497f7e8d3a6a82bfbe9f2"
server_key_iv: "3064615d284ff2df4034f59cc9f5081f"
server_key_version: "encryptionkey@0"
I20260809 00:32:47.551482 23372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:47.552284 23387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:47.552487 23372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:47.552603 23372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "ee0b8e4020d44b59938f8ab4f47282c5"
format_stamp: "Formatted at 2026-08-09 00:32:47 on dist-test-slave-zpfg"
server_key: "d8bf8376575497f7e8d3a6a82bfbe9f2"
server_key_iv: "3064615d284ff2df4034f59cc9f5081f"
server_key_version: "encryptionkey@0"
I20260809 00:32:47.552685 23372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:47.568202 23372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:47.570164 23372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:47.570322 23372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:47.575042 23372 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:33743
I20260809 00:32:47.575112 23449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:33743 every 8 connection(s)
I20260809 00:32:47.575448 23372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:32:47.576092 23450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:47.576979 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 23372
I20260809 00:32:47.577071 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:32:47.577262 21671 external_mini_cluster.cc:1428] Setting key f295a95c7d7ebdddc2f98c8201d1c3d8
I20260809 00:32:47.579378 23450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5: Bootstrap starting.
I20260809 00:32:47.579984 23450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:47.580276 23450 log.cc:824] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:47.580950 23450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5: No bootstrap required, opened a new log
I20260809 00:32:47.582810 23450 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b8e4020d44b59938f8ab4f47282c5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33743 } }
I20260809 00:32:47.582968 23450 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:47.583027 23450 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee0b8e4020d44b59938f8ab4f47282c5, State: Initialized, Role: FOLLOWER
I20260809 00:32:47.583160 23450 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b8e4020d44b59938f8ab4f47282c5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33743 } }
I20260809 00:32:47.583259 23450 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:32:47.583292 23450 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:32:47.583348 23450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 00:32:47 dist-test-slave-zpfg krb5kdc[23359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:47.584059 23450 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b8e4020d44b59938f8ab4f47282c5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33743 } }
I20260809 00:32:47.584179 23450 leader_election.cc:302] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee0b8e4020d44b59938f8ab4f47282c5; no voters:
I20260809 00:32:47.584357 23450 leader_election.cc:288] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:32:47.584400 23455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:47.584561 23455 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [term 1 LEADER]: Becoming Leader. State: Replica: ee0b8e4020d44b59938f8ab4f47282c5, State: Running, Role: LEADER
I20260809 00:32:47.584633 23450 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:32:47.584656 23455 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b8e4020d44b59938f8ab4f47282c5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33743 } }
I20260809 00:32:47.585038 23455 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee0b8e4020d44b59938f8ab4f47282c5. Latest consensus state: current_term: 1 leader_uuid: "ee0b8e4020d44b59938f8ab4f47282c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b8e4020d44b59938f8ab4f47282c5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33743 } } }
I20260809 00:32:47.585124 23455 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:47.585286 23457 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee0b8e4020d44b59938f8ab4f47282c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0b8e4020d44b59938f8ab4f47282c5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33743 } } }
I20260809 00:32:47.585364 23457 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:47.585487 23462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:32:47.586208 23462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:32:47.588033 23462 catalog_manager.cc:1347] Generated new cluster ID: 176c63316a614ac2928c8a9b0c6c159d
I20260809 00:32:47.588083 23462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:32:47.588665 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:47.577803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48964 (local address 127.21.41.254:33743)
0809 00:32:47.577982 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:47.577989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:47.578014 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:32:47.578571 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:47.578580 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:47.578629 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:47.578799 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:47.579409 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.580476 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.581283 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.581693 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.581818 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:47.584403 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:47.584407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:47.584408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:47.584420 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:47.586682 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:47.587086 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:47.587089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:47.587090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:47.587192 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:47.587542 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:47.587543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:47.587543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:47.587822 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:47.587965 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:47.588328 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 00:32:47.588469 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260809 00:32:47.597014 23462 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:32:47.597604 23462 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:32:47.603262 23462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5: Generated new TSK 0
I20260809 00:32:47.603442 23462 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:32:47.635756 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:33743
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:47.751665 23476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:47.751893 23476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:47.751938 23476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:47.753129 23476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:47.753196 23476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:47.753232 23476 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:32:47.753283 23476 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:32:47.753315 23476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:32:47.754904 23476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.41.254:33743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:47.755160 23476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:47.755457 23476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:47.761325 23484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:47.761386 23481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:47.761302 23482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:47.761579 23476 server_base.cc:1034] running on GCE node
I20260809 00:32:47.761873 23476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:47.762118 23476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:47.763255 23476 hybrid_clock.cc:648] HybridClock initialized: now 1786235567763221 us; error 47 us; skew 500 ppm
Aug 09 00:32:47 dist-test-slave-zpfg krb5kdc[23359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235567, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:47.765774 23476 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:32:47.766161 23476 webserver.cc:460] Webserver started at http://127.21.41.193:37723/ using document root <none> and password file <none>
I20260809 00:32:47.766386 23476 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:47.766445 23476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:47.766534 23476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:47.767519 23476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "5aaec76af4d4476ab475c33124a301db"
format_stamp: "Formatted at 2026-08-09 00:32:47 on dist-test-slave-zpfg"
server_key: "731a10ec7f3bca7b5fe6b0e9cd22abdf"
server_key_iv: "546cd33570fc2073e0de3033b2c47088"
server_key_version: "encryptionkey@0"
I20260809 00:32:47.767818 23476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "5aaec76af4d4476ab475c33124a301db"
format_stamp: "Formatted at 2026-08-09 00:32:47 on dist-test-slave-zpfg"
server_key: "731a10ec7f3bca7b5fe6b0e9cd22abdf"
server_key_iv: "546cd33570fc2073e0de3033b2c47088"
server_key_version: "encryptionkey@0"
I20260809 00:32:47.769062 23476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:32:47.769760 23491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:47.769909 23476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:47.769994 23476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "5aaec76af4d4476ab475c33124a301db"
format_stamp: "Formatted at 2026-08-09 00:32:47 on dist-test-slave-zpfg"
server_key: "731a10ec7f3bca7b5fe6b0e9cd22abdf"
server_key_iv: "546cd33570fc2073e0de3033b2c47088"
server_key_version: "encryptionkey@0"
I20260809 00:32:47.770046 23476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:47.803507 23476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:47.805588 23476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:47.805701 23476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:47.806221 23476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:47.806262 23476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:47.806285 23476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:47.806357 23476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:32:47 dist-test-slave-zpfg krb5kdc[23359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235567, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:47.813593 23476 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:36947
I20260809 00:32:47.813655 23606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:36947 every 8 connection(s)
I20260809 00:32:47.813956 23476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:32:47.817278 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:47.806886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:59253 (local address 127.21.41.254:33743)
0809 00:32:47.806943 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:47.806946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:47.807206 (+ 260us) server_negotiation.cc:407] Connection header received
0809 00:32:47.807878 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:47.807880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:47.807895 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:47.807970 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:47.809009 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.809867 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.811231 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.811575 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.811658 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:47.814124 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:47.814125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:47.814126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:47.814134 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:47.815764 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:47.816220 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:47.816222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:47.816222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:47.816287 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:47.816560 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:47.816561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:47.816562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:47.816696 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:47.816784 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:47.817079 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 00:32:47.817176 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:32:47.817490 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:47.806880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:33743 (local address 127.21.41.193:59253)
0809 00:32:47.807070 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:47.807114 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:47.807722 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:47.808059 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:47.808063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:47.808473 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:47.808893 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:47.808901 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.809975 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.809976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:47.811143 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:47.811146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.811663 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.811664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:47.811769 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:47.812343 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:47.812352 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:47.814015 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:47.815877 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:47.815884 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:47.815887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:47.816134 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:47.816361 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:47.816362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:47.816362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:47.816484 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:47.816774 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:47.816781 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:47.816931 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 00:32:47.817051 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 00:32:47.817233 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260809 00:32:47.818012 23607 heartbeater.cc:347] Connected to a master server at 127.21.41.254:33743
I20260809 00:32:47.818095 23607 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:47.818282 23607 heartbeater.cc:511] Master 127.21.41.254:33743 requested a full tablet report, sending...
I20260809 00:32:47.818847 23404 ts_manager.cc:194] Registered new tserver with Master: 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947)
I20260809 00:32:47.819582 23404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:59253
I20260809 00:32:47.822170 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 23476
I20260809 00:32:47.822252 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:32:47.822470 21671 external_mini_cluster.cc:1428] Setting key 59303ac65511e05175cc9ac3e70881f5
I20260809 00:32:47.825194 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:47.819323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:45151 (local address 127.21.41.254:33743)
0809 00:32:47.819364 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:47.819365 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:47.819465 (+ 100us) server_negotiation.cc:407] Connection header received
0809 00:32:47.819505 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:47.819506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:47.819516 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:47.819582 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:47.819944 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.820418 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.821148 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.821379 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.821459 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:47.822545 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:47.822546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:47.822547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:47.822554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:47.824024 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:47.824425 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:47.824426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:47.824427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:47.824488 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:47.824714 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:47.824715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:47.824715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:47.824839 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:47.824896 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:47.825076 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 00:32:47.825137 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:32:47.825320 23613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:47.819260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:33743 (local address 127.21.41.193:45151)
0809 00:32:47.819387 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:47.819398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:47.819459 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:47.819592 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:47.819593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:47.819817 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:47.819874 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:47.819876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.820512 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.820513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:47.821039 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:47.821040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.821469 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.821470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:47.821556 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:47.822040 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:47.822046 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:47.822455 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:47.824147 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:47.824148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:47.824149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:47.824332 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:47.824576 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:47.824577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:47.824578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:47.824654 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:47.824911 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:47.824913 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:47.824969 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 00:32:47.825051 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 00:32:47.825179 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:32:47.849026 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:33743
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:47.937968 23616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:47.938174 23616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:47.938199 23616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:47.939404 23616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:47.939450 23616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:47.939471 23616 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:32:47.939483 23616 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:32:47.939500 23616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:32:47.940996 23616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.41.254:33743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:47.941190 23616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:47.941367 23616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:47.946439 23622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:47.946547 23616 server_base.cc:1034] running on GCE node
W20260809 00:32:47.946444 23624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:47.946475 23621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:47.946883 23616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:47.947093 23616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:47.948226 23616 hybrid_clock.cc:648] HybridClock initialized: now 1786235567948220 us; error 30 us; skew 500 ppm
Aug 09 00:32:47 dist-test-slave-zpfg krb5kdc[23359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235567, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:47.950822 23616 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:32:47.951313 23616 webserver.cc:460] Webserver started at http://127.21.41.194:39627/ using document root <none> and password file <none>
I20260809 00:32:47.951550 23616 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:47.951596 23616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:47.951758 23616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:47.952823 23616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "96f72c86530f4c868f317a250a3cfe85"
format_stamp: "Formatted at 2026-08-09 00:32:47 on dist-test-slave-zpfg"
server_key: "69bcc5638ffd656a48a5d68bc2553b31"
server_key_iv: "182e632c4f9934c098e221d13aad6b05"
server_key_version: "encryptionkey@0"
I20260809 00:32:47.953194 23616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "96f72c86530f4c868f317a250a3cfe85"
format_stamp: "Formatted at 2026-08-09 00:32:47 on dist-test-slave-zpfg"
server_key: "69bcc5638ffd656a48a5d68bc2553b31"
server_key_iv: "182e632c4f9934c098e221d13aad6b05"
server_key_version: "encryptionkey@0"
I20260809 00:32:47.954586 23616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:32:47.955425 23631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:47.955647 23616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:47.955749 23616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "96f72c86530f4c868f317a250a3cfe85"
format_stamp: "Formatted at 2026-08-09 00:32:47 on dist-test-slave-zpfg"
server_key: "69bcc5638ffd656a48a5d68bc2553b31"
server_key_iv: "182e632c4f9934c098e221d13aad6b05"
server_key_version: "encryptionkey@0"
I20260809 00:32:47.955821 23616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:47.967406 23616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:47.969703 23616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:47.969861 23616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:47.970455 23616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:47.970496 23616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:47.970520 23616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:47.970531 23616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:32:47 dist-test-slave-zpfg krb5kdc[23359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235567, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:47.978214 23616 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:45017
I20260809 00:32:47.978293 23746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:45017 every 8 connection(s)
I20260809 00:32:47.978621 23616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:32:47.981127 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:47.970892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:36623 (local address 127.21.41.254:33743)
0809 00:32:47.970963 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:47.970967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:47.971191 (+ 224us) server_negotiation.cc:407] Connection header received
0809 00:32:47.971660 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:47.971662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:47.971677 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:47.971766 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:47.972382 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.973108 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.974775 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.974990 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.975075 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:47.978373 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:47.978375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:47.978376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:47.978385 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:47.979910 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:47.980309 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:47.980310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:47.980310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:47.980358 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:47.980537 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:47.980537 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:47.980538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:47.980634 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:47.980681 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:47.980994 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 00:32:47.981060 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:32:47.981302 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:47.970878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:33743 (local address 127.21.41.194:36623)
0809 00:32:47.971047 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:47.971086 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:47.971567 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:47.971826 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:47.971829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:47.972061 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:47.972275 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:47.972279 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.973211 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.973212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:47.974682 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:47.974684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.975105 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.975107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:47.975233 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:47.975829 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:47.975840 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:47.978240 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:47.979996 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:47.980001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:47.980003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:47.980238 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:47.980400 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:47.980401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:47.980401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:47.980486 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:47.980740 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:47.980745 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:47.980862 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 00:32:47.980963 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 00:32:47.981083 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":105,"threads_started":1}
I20260809 00:32:47.981868 23747 heartbeater.cc:347] Connected to a master server at 127.21.41.254:33743
I20260809 00:32:47.981947 23747 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:47.982133 23747 heartbeater.cc:511] Master 127.21.41.254:33743 requested a full tablet report, sending...
I20260809 00:32:47.982563 23403 ts_manager.cc:194] Registered new tserver with Master: 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194:45017)
I20260809 00:32:47.983170 23403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:36623
I20260809 00:32:47.984784 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 23616
I20260809 00:32:47.984856 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:32:47.985044 21671 external_mini_cluster.cc:1428] Setting key 4396ef49a5d74f40628ffca1e87f111b
I20260809 00:32:47.989084 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:47.982958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:33743 (local address 127.21.41.194:59125)
0809 00:32:47.983099 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:47.983111 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:47.983179 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:47.983350 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:47.983351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:47.983552 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:47.983615 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:47.983616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.984396 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.984397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:47.985170 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:47.985173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.985582 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.985583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:47.985661 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:47.986114 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:47.986118 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:47.986475 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:47.988055 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:47.988056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:47.988056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:47.988225 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:47.988440 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:47.988441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:47.988441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:47.988498 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:47.988842 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:47.988843 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:47.988882 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 00:32:47.988944 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:32:47.989018 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260809 00:32:47.989204 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:47.983045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:59125 (local address 127.21.41.254:33743)
0809 00:32:47.983084 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:47.983086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:47.983182 (+ 96us) server_negotiation.cc:407] Connection header received
0809 00:32:47.983255 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:47.983256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:47.983266 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:47.983319 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:47.983696 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.984313 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.985277 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:47.985514 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:47.985564 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:47.986558 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:47.986560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:47.986560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:47.986568 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:47.987959 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:47.988317 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:47.988318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:47.988319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:47.988375 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:47.988572 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:47.988572 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:47.988573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:47.988772 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:47.988841 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:47.988990 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 00:32:47.989072 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:32:48.013549 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:33743
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:48.107535 23756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:48.107744 23756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:48.107769 23756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:48.108907 23756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:48.108945 23756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:48.108965 23756 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:32:48.108978 23756 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:32:48.108995 23756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:32:48.110440 23756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.41.254:33743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:48.110631 23756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:48.110819 23756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:48.115935 23764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:48.115931 23761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:48.116020 23756 server_base.cc:1034] running on GCE node
W20260809 00:32:48.115955 23762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:48.116358 23756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:48.116572 23756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:48.117720 23756 hybrid_clock.cc:648] HybridClock initialized: now 1786235568117680 us; error 55 us; skew 500 ppm
Aug 09 00:32:48 dist-test-slave-zpfg krb5kdc[23359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235568, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:48.120388 23756 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:32:48.120791 23756 webserver.cc:460] Webserver started at http://127.21.41.195:37881/ using document root <none> and password file <none>
I20260809 00:32:48.120973 23756 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:48.121014 23756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:48.121110 23756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:48.122157 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "059874e6a7894fe5b503b7f826fb05b9"
format_stamp: "Formatted at 2026-08-09 00:32:48 on dist-test-slave-zpfg"
server_key: "961a462fb91b7f98dd4fcba30421e948"
server_key_iv: "6a683a785981110d43fe73b9dc07f05d"
server_key_version: "encryptionkey@0"
I20260809 00:32:48.122552 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "059874e6a7894fe5b503b7f826fb05b9"
format_stamp: "Formatted at 2026-08-09 00:32:48 on dist-test-slave-zpfg"
server_key: "961a462fb91b7f98dd4fcba30421e948"
server_key_iv: "6a683a785981110d43fe73b9dc07f05d"
server_key_version: "encryptionkey@0"
I20260809 00:32:48.124145 23756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:32:48.124848 23771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:48.125032 23756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:32:48.125119 23756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "059874e6a7894fe5b503b7f826fb05b9"
format_stamp: "Formatted at 2026-08-09 00:32:48 on dist-test-slave-zpfg"
server_key: "961a462fb91b7f98dd4fcba30421e948"
server_key_iv: "6a683a785981110d43fe73b9dc07f05d"
server_key_version: "encryptionkey@0"
I20260809 00:32:48.125173 23756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:48.133471 23756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:48.135409 23756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:48.135515 23756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:48.135982 23756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:48.136018 23756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:48.136044 23756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:48.136092 23756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:48.142705 23756 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:44027
I20260809 00:32:48.142765 23886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:44027 every 8 connection(s)
I20260809 00:32:48.143107 23756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:32:48 dist-test-slave-zpfg krb5kdc[23359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235568, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:48.146570 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.136522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:35621 (local address 127.21.41.254:33743)
0809 00:32:48.136564 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.136566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.136833 (+ 267us) server_negotiation.cc:407] Connection header received
0809 00:32:48.137360 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.137361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.137373 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.137433 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:48.138291 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.139196 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.140861 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.141078 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.141176 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.143852 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:48.143853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:48.143854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:48.143861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:48.145003 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:48.145435 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:48.145437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:48.145437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:48.145503 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:48.145762 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:48.145763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:48.145763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:48.145903 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:48.145969 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.146332 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.146477 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:32:48.146714 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.136525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:33743 (local address 127.21.41.195:35621)
0809 00:32:48.136714 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.136748 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.137249 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.137503 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.137507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.137871 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:48.138169 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.138175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.139312 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.139313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.140781 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.140784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.141198 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.141200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.141346 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.141963 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:48.141973 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:48.143740 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:48.145119 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:48.145125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:48.145127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:48.145351 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:48.145569 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:48.145570 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:48.145571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:48.145686 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:48.145996 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:48.146001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:48.146175 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.146295 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.146476 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
I20260809 00:32:48.147351 23887 heartbeater.cc:347] Connected to a master server at 127.21.41.254:33743
I20260809 00:32:48.147432 23887 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:48.147614 23887 heartbeater.cc:511] Master 127.21.41.254:33743 requested a full tablet report, sending...
I20260809 00:32:48.148362 23403 ts_manager.cc:194] Registered new tserver with Master: 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195:44027)
I20260809 00:32:48.148774 23403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:35621
I20260809 00:32:48.149150 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 23756
I20260809 00:32:48.149217 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:32:48.149399 21671 external_mini_cluster.cc:1428] Setting key bc306c05933155b2f765e1892e0bc362
I20260809 00:32:48.150733 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:32:48.154857 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.148469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:51559 (local address 127.21.41.254:33743)
0809 00:32:48.148912 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.148913 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.148922 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:32:48.148961 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.148962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.148972 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.149030 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:48.149485 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.150035 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.150950 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.151287 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.151360 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.152225 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:48.152226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:48.152227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:48.152234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:48.153655 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:48.154036 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:48.154037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:48.154038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:48.154114 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:48.154328 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:48.154329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:48.154329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:48.154479 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:48.154533 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.154733 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.154797 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425}
I20260809 00:32:48.154971 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.148403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:33743 (local address 127.21.41.195:51559)
0809 00:32:48.148534 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.148545 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.148597 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.149058 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.149059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.149324 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:48.149389 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.149391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.150121 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.150121 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.150882 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.150884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.151364 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.151365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.151419 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.151848 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:48.151852 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:48.152109 (+ 257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:48.153763 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:48.153764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:48.153764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:48.153940 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:48.154192 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:48.154193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:48.154193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:48.154267 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:48.154545 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:48.154546 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:48.154599 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.154675 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.154838 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
Aug 09 00:32:48 dist-test-slave-zpfg krb5kdc[23359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:32:48 dist-test-slave-zpfg krb5kdc[23359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:48.175036 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.168375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.21.41.254:33743)
0809 00:32:48.168440 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.168442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.168465 (+ 23us) server_negotiation.cc:407] Connection header received
0809 00:32:48.168535 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.168536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.168544 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.168596 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:48.169134 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.169614 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.170325 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.170561 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.170613 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.172836 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:48.172838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:48.172838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:48.172845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:48.174030 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:48.174372 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:48.174373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:48.174373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:48.174432 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:48.174600 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:48.174601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:48.174601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:48.174710 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:48.174789 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.174907 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.174968 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:32:48.176705 23403 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:32:48.178426 23403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:32:48.185374 23906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.180930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.21.41.195:44027)
0809 00:32:48.181451 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.181456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.181466 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:32:48.181514 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.181516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.181531 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.181785 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:48.181848 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.183016 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.184186 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.184927 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.185005 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.185126 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.185239 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.185277 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":85,"threads_started":1}
I20260809 00:32:48.185369 23908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.180873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:36947 (local address 127.0.0.1:54938)
0809 00:32:48.181164 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.181177 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.181234 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.181576 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.181579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.181582 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:48.181650 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.181651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.182739 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.182740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.184175 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.184178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.185068 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.185068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.185124 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.185140 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.185203 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.185277 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":88,"threads_started":1}
I20260809 00:32:48.185372 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.180846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:44027 (local address 127.0.0.1:48218)
0809 00:32:48.181031 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.181076 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.181141 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.181637 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.181638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.181640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:48.181706 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.181707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.183128 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.183130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.184083 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.184085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.185037 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.185038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.185126 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.185140 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.185214 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.185307 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":49,"thread_start_us":61,"threads_started":1}
I20260809 00:32:48.186314 23821 tablet_service.cc:1467] Processing CreateTablet for tablet 22964da0834d441a8ecb127db927c1b9 (DEFAULT_TABLE table=test-table [id=56acb07e34674869afd0d349033cc8c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:48.186648 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.180981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.21.41.193:36947)
0809 00:32:48.181168 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.181173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.181303 (+ 130us) server_negotiation.cc:407] Connection header received
0809 00:32:48.181354 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.181356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.181373 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.181447 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:48.181731 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.182655 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.184246 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.184962 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.186390 (+ 1428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.186463 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.186487 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.186530 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":78,"threads_started":1}
I20260809 00:32:48.186671 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.181057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:45017 (local address 127.0.0.1:55122)
0809 00:32:48.181273 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.181282 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.181325 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.181647 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.181648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.181649 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:48.181706 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.181707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.183095 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.183096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.184178 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.184180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.186435 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.186436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.186500 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.186504 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.186557 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.186599 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":67,"threads_started":1}
I20260809 00:32:48.186782 23821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22964da0834d441a8ecb127db927c1b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:48.186966 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.181286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55122 (local address 127.21.41.194:45017)
0809 00:32:48.181483 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.181486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.181495 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:32:48.181534 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.181535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.181550 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.181812 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:48.181869 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.183020 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.185096 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.186363 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.186437 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.186492 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.186823 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.186864 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":100,"threads_started":1}
I20260809 00:32:48.187259 23541 tablet_service.cc:1467] Processing CreateTablet for tablet 22964da0834d441a8ecb127db927c1b9 (DEFAULT_TABLE table=test-table [id=56acb07e34674869afd0d349033cc8c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:48.187521 23541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22964da0834d441a8ecb127db927c1b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:48.187528 23681 tablet_service.cc:1467] Processing CreateTablet for tablet 22964da0834d441a8ecb127db927c1b9 (DEFAULT_TABLE table=test-table [id=56acb07e34674869afd0d349033cc8c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:48.187855 23681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22964da0834d441a8ecb127db927c1b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:48.190174 23910 tablet_bootstrap.cc:492] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: Bootstrap starting.
I20260809 00:32:48.191077 23910 tablet_bootstrap.cc:654] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:48.191344 23910 log.cc:824] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:48.191362 23912 tablet_bootstrap.cc:492] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: Bootstrap starting.
I20260809 00:32:48.191639 23911 tablet_bootstrap.cc:492] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: Bootstrap starting.
I20260809 00:32:48.192027 23912 tablet_bootstrap.cc:654] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:48.192067 23910 tablet_bootstrap.cc:492] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: No bootstrap required, opened a new log
I20260809 00:32:48.192148 23910 ts_tablet_manager.cc:1397] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:32:48.192335 23911 tablet_bootstrap.cc:654] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:48.192394 23912 log.cc:824] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:48.192760 23911 log.cc:824] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:48.193157 23912 tablet_bootstrap.cc:492] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: No bootstrap required, opened a new log
I20260809 00:32:48.193233 23912 ts_tablet_manager.cc:1397] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:32:48.193481 23911 tablet_bootstrap.cc:492] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: No bootstrap required, opened a new log
I20260809 00:32:48.193562 23911 ts_tablet_manager.cc:1397] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:32:48.194406 23910 raft_consensus.cc:348] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } }
I20260809 00:32:48.194579 23910 raft_consensus.cc:374] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:48.194614 23910 raft_consensus.cc:729] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059874e6a7894fe5b503b7f826fb05b9, State: Initialized, Role: FOLLOWER
I20260809 00:32:48.194743 23910 consensus_queue.cc:260] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } }
I20260809 00:32:48.195030 23887 heartbeater.cc:503] Master 127.21.41.254:33743 was elected leader, sending a full tablet report...
I20260809 00:32:48.195281 23910 ts_tablet_manager.cc:1428] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:32:48.195353 23912 raft_consensus.cc:348] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } }
I20260809 00:32:48.195495 23912 raft_consensus.cc:374] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:48.195533 23912 raft_consensus.cc:729] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f72c86530f4c868f317a250a3cfe85, State: Initialized, Role: FOLLOWER
I20260809 00:32:48.195654 23912 consensus_queue.cc:260] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } }
I20260809 00:32:48.195758 23911 raft_consensus.cc:348] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } }
I20260809 00:32:48.195919 23911 raft_consensus.cc:374] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:48.195925 23912 ts_tablet_manager.cc:1428] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:32:48.195966 23911 raft_consensus.cc:729] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aaec76af4d4476ab475c33124a301db, State: Initialized, Role: FOLLOWER
I20260809 00:32:48.196043 23747 heartbeater.cc:503] Master 127.21.41.254:33743 was elected leader, sending a full tablet report...
I20260809 00:32:48.196097 23911 consensus_queue.cc:260] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } }
I20260809 00:32:48.196352 23911 ts_tablet_manager.cc:1428] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:32:48.196398 23607 heartbeater.cc:503] Master 127.21.41.254:33743 was elected leader, sending a full tablet report...
W20260809 00:32:48.229229 23748 tablet.cc:2369] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:32:48.252030 23917 raft_consensus.cc:482] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:32:48.252157 23917 raft_consensus.cc:504] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } }
I20260809 00:32:48.252532 23917 leader_election.cc:288] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195:44027), 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947)
I20260809 00:32:48.256582 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.252857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:56297 (local address 127.21.41.193:36947)
0809 00:32:48.252923 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.252927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.252937 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:32:48.252966 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.252967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.252977 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.253035 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:48.253276 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.254023 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.255595 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.256198 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.256289 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.256297 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.256465 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.256501 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:32:48.256573 23906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.252796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:46647 (local address 127.21.41.195:44027)
0809 00:32:48.252837 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.252840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.253107 (+ 267us) server_negotiation.cc:407] Connection header received
0809 00:32:48.253147 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.253148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.253155 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.253197 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:48.253380 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.254059 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.255601 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.256198 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.256277 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.256287 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.256465 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.256502 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:32:48.256578 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.252771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:36947 (local address 127.21.41.194:56297)
0809 00:32:48.252829 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.252841 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.252884 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.253066 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.253067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.253069 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:48.253146 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.253148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.254128 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.254129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.255498 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.255501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.256300 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.256301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.256397 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.256399 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.256450 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.256482 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":78}
I20260809 00:32:48.256590 23920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.252793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:44027 (local address 127.21.41.194:46647)
0809 00:32:48.253002 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.253010 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.253086 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.253233 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.253234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.253236 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:48.253310 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.253311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.254129 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.254130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.255498 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.255501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.256292 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.256293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.256372 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.256377 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.256439 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.256482 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":70,"thread_start_us":135,"threads_started":1}
I20260809 00:32:48.257058 23561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22964da0834d441a8ecb127db927c1b9" candidate_uuid: "96f72c86530f4c868f317a250a3cfe85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aaec76af4d4476ab475c33124a301db" is_pre_election: true
I20260809 00:32:48.257058 23841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22964da0834d441a8ecb127db927c1b9" candidate_uuid: "96f72c86530f4c868f317a250a3cfe85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "059874e6a7894fe5b503b7f826fb05b9" is_pre_election: true
I20260809 00:32:48.257225 23841 raft_consensus.cc:2454] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f72c86530f4c868f317a250a3cfe85 in term 0.
I20260809 00:32:48.257225 23561 raft_consensus.cc:2454] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f72c86530f4c868f317a250a3cfe85 in term 0.
I20260809 00:32:48.257476 23634 leader_election.cc:302] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aaec76af4d4476ab475c33124a301db, 96f72c86530f4c868f317a250a3cfe85; no voters:
I20260809 00:32:48.257659 23917 raft_consensus.cc:2781] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:32:48.257714 23917 raft_consensus.cc:482] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:32:48.257731 23917 raft_consensus.cc:3037] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:48.258819 23917 raft_consensus.cc:504] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } }
I20260809 00:32:48.258965 23917 leader_election.cc:288] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [CANDIDATE]: Term 1 election: Requested vote from peers 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195:44027), 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947)
I20260809 00:32:48.259094 23841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22964da0834d441a8ecb127db927c1b9" candidate_uuid: "96f72c86530f4c868f317a250a3cfe85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "059874e6a7894fe5b503b7f826fb05b9"
I20260809 00:32:48.259150 23561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22964da0834d441a8ecb127db927c1b9" candidate_uuid: "96f72c86530f4c868f317a250a3cfe85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aaec76af4d4476ab475c33124a301db"
I20260809 00:32:48.259176 23841 raft_consensus.cc:3037] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:48.259232 23561 raft_consensus.cc:3037] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:48.260067 23841 raft_consensus.cc:2454] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f72c86530f4c868f317a250a3cfe85 in term 1.
I20260809 00:32:48.260067 23561 raft_consensus.cc:2454] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f72c86530f4c868f317a250a3cfe85 in term 1.
I20260809 00:32:48.260231 23634 leader_election.cc:302] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aaec76af4d4476ab475c33124a301db, 96f72c86530f4c868f317a250a3cfe85; no voters:
I20260809 00:32:48.260346 23917 raft_consensus.cc:2781] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:48.260517 23917 raft_consensus.cc:686] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 1 LEADER]: Becoming Leader. State: Replica: 96f72c86530f4c868f317a250a3cfe85, State: Running, Role: LEADER
I20260809 00:32:48.260639 23917 consensus_queue.cc:237] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } }
I20260809 00:32:48.261523 23404 catalog_manager.cc:5495] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194). New cstate: current_term: 1 leader_uuid: "96f72c86530f4c868f317a250a3cfe85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } health_report { overall_health: UNKNOWN } } }
Aug 09 00:32:48 dist-test-slave-zpfg krb5kdc[23359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235567, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:48.305780 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.299015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:33743 (local address 127.0.0.1:48970)
0809 00:32:48.299169 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.299184 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.299249 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.299373 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.299374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.299674 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:48.299723 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.299725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.300343 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.300344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.300877 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.300878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.301176 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.301176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.301215 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.301697 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:48.301702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:48.303167 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:48.304752 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:48.304759 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:48.304761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:48.304947 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:48.305178 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:48.305179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:48.305179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:48.305236 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:48.305487 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:48.305493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:48.305537 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.305599 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.305694 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260809 00:32:48.305877 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.299114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48970 (local address 127.21.41.254:33743)
0809 00:32:48.299192 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.299193 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.299246 (+ 53us) server_negotiation.cc:407] Connection header received
0809 00:32:48.299294 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.299295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.299305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.299354 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:48.299805 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.300278 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.300941 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.301114 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.301162 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.303254 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:48.303255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:48.303256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:48.303262 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:48.304666 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:48.305031 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:48.305032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:48.305033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:48.305104 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:48.305295 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:48.305300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:48.305300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:48.305408 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:48.305462 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.305645 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.305745 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":53}
I20260809 00:32:48.307965 23404 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:48970:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 00:32:48.308218 23404 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:32:48.309998 23541 tablet_service.cc:1467] Processing CreateTablet for tablet eef3c238052340eea1d71be29772b97c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=72b54552dea946a690ee44d3a22eee14]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:32:48.310045 23681 tablet_service.cc:1467] Processing CreateTablet for tablet eef3c238052340eea1d71be29772b97c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=72b54552dea946a690ee44d3a22eee14]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:32:48.310202 23541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef3c238052340eea1d71be29772b97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:48.310209 23681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef3c238052340eea1d71be29772b97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:48.311684 23912 tablet_bootstrap.cc:492] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85: Bootstrap starting.
I20260809 00:32:48.311744 23911 tablet_bootstrap.cc:492] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: Bootstrap starting.
I20260809 00:32:48.311872 23821 tablet_service.cc:1467] Processing CreateTablet for tablet eef3c238052340eea1d71be29772b97c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=72b54552dea946a690ee44d3a22eee14]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:32:48.312033 23821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef3c238052340eea1d71be29772b97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:48.312176 23912 tablet_bootstrap.cc:654] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:48.312592 23911 tablet_bootstrap.cc:654] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:48.312882 23912 tablet_bootstrap.cc:492] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85: No bootstrap required, opened a new log
I20260809 00:32:48.312966 23912 ts_tablet_manager.cc:1397] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:48.313110 23912 raft_consensus.cc:348] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.313201 23912 raft_consensus.cc:374] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:48.313227 23912 raft_consensus.cc:729] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f72c86530f4c868f317a250a3cfe85, State: Initialized, Role: FOLLOWER
I20260809 00:32:48.313283 23912 consensus_queue.cc:260] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.313345 23911 tablet_bootstrap.cc:492] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: No bootstrap required, opened a new log
I20260809 00:32:48.313381 23912 ts_tablet_manager.cc:1428] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:48.313385 23911 ts_tablet_manager.cc:1397] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:32:48.313437 23917 tablet_replica.cc:439] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } }
I20260809 00:32:48.313539 23917 tablet_replica.cc:442] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:48.313525 23911 raft_consensus.cc:348] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.313594 23911 raft_consensus.cc:374] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:48.313616 23911 raft_consensus.cc:729] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aaec76af4d4476ab475c33124a301db, State: Initialized, Role: FOLLOWER
I20260809 00:32:48.313680 23911 consensus_queue.cc:260] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.313768 23911 ts_tablet_manager.cc:1428] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:48.313844 23919 tablet_replica.cc:439] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } }
I20260809 00:32:48.313916 23919 tablet_replica.cc:442] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:48.313938 23910 tablet_bootstrap.cc:492] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9: Bootstrap starting.
I20260809 00:32:48.314673 23910 tablet_bootstrap.cc:654] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9: Neither blocks nor log segments found. Creating new log.
W20260809 00:32:48.314704 23608 tablet.cc:2369] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:32:48.315409 23910 tablet_bootstrap.cc:492] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9: No bootstrap required, opened a new log
I20260809 00:32:48.315455 23910 ts_tablet_manager.cc:1397] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:32:48.315588 23910 raft_consensus.cc:348] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.315647 23910 raft_consensus.cc:374] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:48.315662 23910 raft_consensus.cc:729] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059874e6a7894fe5b503b7f826fb05b9, State: Initialized, Role: FOLLOWER
I20260809 00:32:48.315727 23910 consensus_queue.cc:260] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.315830 23910 ts_tablet_manager.cc:1428] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:48.315915 23916 tablet_replica.cc:439] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } }
I20260809 00:32:48.315997 23916 tablet_replica.cc:442] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:48.371142 23919 raft_consensus.cc:482] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:32:48.371275 23919 raft_consensus.cc:504] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.371645 23919 leader_election.cc:288] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194:45017), 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195:44027)
I20260809 00:32:48.375594 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.371941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:58901 (local address 127.21.41.194:45017)
0809 00:32:48.372016 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.372017 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.372047 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:32:48.372112 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.372114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.372124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.372193 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:48.372356 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.373120 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.374656 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.375219 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.375290 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.375297 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.375465 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.375506 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 00:32:48.375594 23906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.372144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:50189 (local address 127.21.41.195:44027)
0809 00:32:48.372188 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.372190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.372199 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:32:48.372299 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.372300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.372306 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.372382 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:48.372545 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.373273 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.374651 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.375204 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.375288 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.375297 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.375475 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.375517 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:32:48.375633 23932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.371848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:45017 (local address 127.21.41.193:58901)
0809 00:32:48.371975 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.371988 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.372045 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.372217 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.372218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.372221 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:48.372286 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.372288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.373207 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.373208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.374558 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.374562 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.375299 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.375300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.375380 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.375385 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.375445 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.375510 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":36,"thread_start_us":79,"threads_started":1}
I20260809 00:32:48.375645 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.372039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:44027 (local address 127.21.41.193:50189)
0809 00:32:48.372150 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.372160 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.372225 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.372397 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.372397 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.372399 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:48.372480 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.372481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.373337 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.373337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.374558 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.374562 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.375296 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.375297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.375385 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.375388 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.375445 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.375510 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":45,"thread_start_us":41,"threads_started":1}
I20260809 00:32:48.376137 23841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef3c238052340eea1d71be29772b97c" candidate_uuid: "5aaec76af4d4476ab475c33124a301db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "059874e6a7894fe5b503b7f826fb05b9" is_pre_election: true
I20260809 00:32:48.376158 23701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef3c238052340eea1d71be29772b97c" candidate_uuid: "5aaec76af4d4476ab475c33124a301db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f72c86530f4c868f317a250a3cfe85" is_pre_election: true
I20260809 00:32:48.376273 23841 raft_consensus.cc:2454] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aaec76af4d4476ab475c33124a301db in term 0.
I20260809 00:32:48.376286 23701 raft_consensus.cc:2454] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aaec76af4d4476ab475c33124a301db in term 0.
I20260809 00:32:48.376467 23493 leader_election.cc:302] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059874e6a7894fe5b503b7f826fb05b9, 5aaec76af4d4476ab475c33124a301db; no voters:
I20260809 00:32:48.376637 23919 raft_consensus.cc:2781] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:32:48.376703 23919 raft_consensus.cc:482] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:32:48.376739 23919 raft_consensus.cc:3037] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:48.377424 23919 raft_consensus.cc:504] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.377573 23919 leader_election.cc:288] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [CANDIDATE]: Term 1 election: Requested vote from peers 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194:45017), 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195:44027)
I20260809 00:32:48.377795 23701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef3c238052340eea1d71be29772b97c" candidate_uuid: "5aaec76af4d4476ab475c33124a301db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f72c86530f4c868f317a250a3cfe85"
I20260809 00:32:48.377789 23841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef3c238052340eea1d71be29772b97c" candidate_uuid: "5aaec76af4d4476ab475c33124a301db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "059874e6a7894fe5b503b7f826fb05b9"
I20260809 00:32:48.377869 23841 raft_consensus.cc:3037] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:48.377869 23701 raft_consensus.cc:3037] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:48.378723 23841 raft_consensus.cc:2454] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aaec76af4d4476ab475c33124a301db in term 1.
I20260809 00:32:48.378722 23701 raft_consensus.cc:2454] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aaec76af4d4476ab475c33124a301db in term 1.
I20260809 00:32:48.378916 23493 leader_election.cc:302] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aaec76af4d4476ab475c33124a301db, 96f72c86530f4c868f317a250a3cfe85; no voters:
I20260809 00:32:48.378996 23919 raft_consensus.cc:2781] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:48.379122 23919 raft_consensus.cc:686] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [term 1 LEADER]: Becoming Leader. State: Replica: 5aaec76af4d4476ab475c33124a301db, State: Running, Role: LEADER
I20260809 00:32:48.379247 23919 consensus_queue.cc:237] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.379704 23934 tablet_replica.cc:439] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: TxnStatusTablet state changed. Reason: New leader 5aaec76af4d4476ab475c33124a301db. Latest consensus state: current_term: 1 leader_uuid: "5aaec76af4d4476ab475c33124a301db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } }
I20260809 00:32:48.379832 23934 tablet_replica.cc:442] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:32:48.380066 23936 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 00:32:48.380182 23936 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 00:32:48.380328 23404 catalog_manager.cc:5495] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5aaec76af4d4476ab475c33124a301db (127.21.41.193). New cstate: current_term: 1 leader_uuid: "5aaec76af4d4476ab475c33124a301db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } health_report { overall_health: UNKNOWN } } }
I20260809 00:32:48.388911 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.386376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54954 (local address 127.21.41.193:36947)
0809 00:32:48.386430 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.386432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.386476 (+ 44us) server_negotiation.cc:407] Connection header received
0809 00:32:48.386575 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.386575 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.386593 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.386655 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:48.386811 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.387316 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.388066 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.388297 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.388376 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.388577 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.388684 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.388774 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.388817 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.388837 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:32:48.388929 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.386258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:36947 (local address 127.0.0.1:54954)
0809 00:32:48.386430 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.386443 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.386497 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.386671 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.386672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.386674 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:32:48.386727 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.386729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.387412 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.387413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.387985 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.387987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.388374 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.388375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.388452 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.388474 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.388752 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.388762 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.388810 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.388859 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260809 00:32:48.390825 23841 raft_consensus.cc:1261] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Refusing update from remote peer 5aaec76af4d4476ab475c33124a301db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:48.390851 23701 raft_consensus.cc:1261] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Refusing update from remote peer 5aaec76af4d4476ab475c33124a301db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:48.391134 23934 consensus_queue.cc:1035] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [LEADER]: Connected to new peer: Peer: permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:48.391304 23919 consensus_queue.cc:1035] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:48.392572 23917 tablet_replica.cc:439] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85: TxnStatusTablet state changed. Reason: New leader 5aaec76af4d4476ab475c33124a301db. Latest consensus state: current_term: 1 leader_uuid: "5aaec76af4d4476ab475c33124a301db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } }
I20260809 00:32:48.392572 23916 tablet_replica.cc:439] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9: TxnStatusTablet state changed. Reason: New leader 5aaec76af4d4476ab475c33124a301db. Latest consensus state: current_term: 1 leader_uuid: "5aaec76af4d4476ab475c33124a301db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } }
I20260809 00:32:48.392665 23917 tablet_replica.cc:442] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:48.392683 23916 tablet_replica.cc:442] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 00:32:48.393734 23888 tablet.cc:2369] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:32:48.393733 23919 tablet_replica.cc:439] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5aaec76af4d4476ab475c33124a301db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } }
I20260809 00:32:48.393806 23919 tablet_replica.cc:442] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:32:48.393849 23919 tablet_replica.cc:439] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5aaec76af4d4476ab475c33124a301db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } }
I20260809 00:32:48.393893 23919 tablet_replica.cc:442] T eef3c238052340eea1d71be29772b97c P 5aaec76af4d4476ab475c33124a301db: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:32:48.394464 23916 tablet_replica.cc:439] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5aaec76af4d4476ab475c33124a301db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } }
I20260809 00:32:48.394547 23916 tablet_replica.cc:442] T eef3c238052340eea1d71be29772b97c P 059874e6a7894fe5b503b7f826fb05b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:48.394793 23917 tablet_replica.cc:439] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5aaec76af4d4476ab475c33124a301db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } } }
I20260809 00:32:48.394870 23917 tablet_replica.cc:442] T eef3c238052340eea1d71be29772b97c P 96f72c86530f4c868f317a250a3cfe85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:32:48.395289 23941 mvcc.cc:205] Tried to move back new op lower bound from 7316420888126115840 to 7316420888082001920. Current Snapshot: MvccSnapshot[applied={T|T < 7316420888126115840}]
I20260809 00:32:48.398720 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.396241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55124 (local address 127.21.41.194:45017)
0809 00:32:48.396284 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.396286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.396295 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:32:48.396375 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.396375 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.396392 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.396456 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:48.396613 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.397080 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.397838 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.398049 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.398133 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.398314 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.398484 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.398579 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.398619 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.398641 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:32:48.403496 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.401137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:42443 (local address 127.21.41.193:36947)
0809 00:32:48.401188 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.401190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.401199 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:32:48.401262 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.401263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.401273 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.401327 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:48.401461 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.401947 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.402776 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.402968 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.403050 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.403256 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.403293 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.403349 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.403408 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.403426 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:32:48.403514 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.401065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:36947 (local address 127.21.41.194:42443)
0809 00:32:48.401140 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.401152 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.401205 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.401329 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.401330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.401331 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:32:48.401388 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.401390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.402032 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.402034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.402688 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.402691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.403052 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.403054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.403131 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.403144 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.403364 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.403366 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.403407 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.403450 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:32:48.405823 23561 raft_consensus.cc:1261] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Refusing update from remote peer 96f72c86530f4c868f317a250a3cfe85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:48.405961 23841 raft_consensus.cc:1261] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Refusing update from remote peer 96f72c86530f4c868f317a250a3cfe85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:48.406114 23921 consensus_queue.cc:1035] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:48.406229 23917 consensus_queue.cc:1035] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:48.408586 23939 mvcc.cc:205] Tried to move back new op lower bound from 7316420888188379136 to 7316420887596339200. Current Snapshot: MvccSnapshot[applied={T|T < 7316420888188379136}]
I20260809 00:32:48.415256 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.412523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:51527 (local address 127.21.41.194:45017)
0809 00:32:48.412563 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.412565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.412711 (+ 146us) server_negotiation.cc:407] Connection header received
0809 00:32:48.412787 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.412788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.412797 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.412840 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:48.412967 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.413437 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.414606 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.414795 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.414850 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.415032 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.415056 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.415089 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.415166 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.415182 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:32:48.415307 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.412485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:45017 (local address 127.21.41.193:51527)
0809 00:32:48.412655 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.412666 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.412723 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.412846 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.412847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.412848 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:32:48.412913 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.412914 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.413520 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.413521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.414523 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.414526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.414869 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.414870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.414938 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.414956 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.415113 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.415114 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.415165 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.415207 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260809 00:32:48.424479 23403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:48.424598 23403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:48.425370 23403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5: Sending DeleteTablet for 3 replicas of tablet 22964da0834d441a8ecb127db927c1b9
I20260809 00:32:48.425669 23821 tablet_service.cc:1514] Processing DeleteTablet for tablet 22964da0834d441a8ecb127db927c1b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:48 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:48218
I20260809 00:32:48.425817 23681 tablet_service.cc:1514] Processing DeleteTablet for tablet 22964da0834d441a8ecb127db927c1b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:48 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:55122
I20260809 00:32:48.425987 23958 tablet_replica.cc:330] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: stopping tablet replica
I20260809 00:32:48.426052 23541 tablet_service.cc:1514] Processing DeleteTablet for tablet 22964da0834d441a8ecb127db927c1b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:48 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:54938
I20260809 00:32:48.426234 23958 raft_consensus.cc:2229] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:32:48.426386 23958 raft_consensus.cc:2258] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:48.426566 23403 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:32:48.426705 23403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:32:48.427850 23541 tablet_service.cc:1467] Processing CreateTablet for tablet 302f9b6919bf4f6290ae50588fdea56d (DEFAULT_TABLE table=test-table [id=79113b44c7f84072b1ecbe2bd3906325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:48.427917 23821 tablet_service.cc:1467] Processing CreateTablet for tablet 302f9b6919bf4f6290ae50588fdea56d (DEFAULT_TABLE table=test-table [id=79113b44c7f84072b1ecbe2bd3906325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:48.427975 23541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302f9b6919bf4f6290ae50588fdea56d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:48.428032 23821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302f9b6919bf4f6290ae50588fdea56d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:48.429387 23911 tablet_bootstrap.cc:492] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db: Bootstrap starting.
I20260809 00:32:48.429494 23910 tablet_bootstrap.cc:492] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9: Bootstrap starting.
I20260809 00:32:48.429883 23910 tablet_bootstrap.cc:654] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:48.429945 23911 tablet_bootstrap.cc:654] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:48.430517 23681 tablet_service.cc:1467] Processing CreateTablet for tablet 302f9b6919bf4f6290ae50588fdea56d (DEFAULT_TABLE table=test-table [id=79113b44c7f84072b1ecbe2bd3906325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:48.430673 23681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302f9b6919bf4f6290ae50588fdea56d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:48.432054 23912 tablet_bootstrap.cc:492] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85: Bootstrap starting.
I20260809 00:32:48.432492 23912 tablet_bootstrap.cc:654] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:48.433148 23958 ts_tablet_manager.cc:1905] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:48.434764 23911 tablet_bootstrap.cc:492] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db: No bootstrap required, opened a new log
I20260809 00:32:48.434783 23912 tablet_bootstrap.cc:492] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85: No bootstrap required, opened a new log
I20260809 00:32:48.434824 23911 ts_tablet_manager.cc:1397] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260809 00:32:48.434839 23912 ts_tablet_manager.cc:1397] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:32:48.435000 23912 raft_consensus.cc:348] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.435073 23912 raft_consensus.cc:374] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:48.435000 23911 raft_consensus.cc:348] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.435093 23912 raft_consensus.cc:729] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f72c86530f4c868f317a250a3cfe85, State: Initialized, Role: FOLLOWER
I20260809 00:32:48.435105 23911 raft_consensus.cc:374] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:48.435128 23911 raft_consensus.cc:729] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aaec76af4d4476ab475c33124a301db, State: Initialized, Role: FOLLOWER
I20260809 00:32:48.435137 23912 consensus_queue.cc:260] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.435175 23911 consensus_queue.cc:260] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.435148 23958 ts_tablet_manager.cc:1918] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:32:48.435230 23912 ts_tablet_manager.cc:1428] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:48.435286 23911 ts_tablet_manager.cc:1428] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:48.436017 23959 tablet_replica.cc:330] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: stopping tablet replica
I20260809 00:32:48.436143 23959 raft_consensus.cc:2229] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:32:48.436261 23959 raft_consensus.cc:2258] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:48.435916 23910 tablet_bootstrap.cc:492] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9: No bootstrap required, opened a new log
I20260809 00:32:48.436743 23910 ts_tablet_manager.cc:1397] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260809 00:32:48.436761 23959 ts_tablet_manager.cc:1905] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:48.436900 23910 raft_consensus.cc:348] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.436973 23910 raft_consensus.cc:374] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:48.436995 23910 raft_consensus.cc:729] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059874e6a7894fe5b503b7f826fb05b9, State: Initialized, Role: FOLLOWER
I20260809 00:32:48.437047 23910 consensus_queue.cc:260] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.437148 23910 ts_tablet_manager.cc:1428] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:48.435272 23958 log.cc:1196] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/22964da0834d441a8ecb127db927c1b9
I20260809 00:32:48.437754 23958 ts_tablet_manager.cc:1939] T 22964da0834d441a8ecb127db927c1b9 P 059874e6a7894fe5b503b7f826fb05b9: Deleting consensus metadata
I20260809 00:32:48.438215 23389 catalog_manager.cc:4841] TS 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195:44027): tablet 22964da0834d441a8ecb127db927c1b9 (table test-table [id=56acb07e34674869afd0d349033cc8c2]) successfully deleted
I20260809 00:32:48.438455 23959 ts_tablet_manager.cc:1918] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:32:48.438598 23959 log.cc:1196] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/22964da0834d441a8ecb127db927c1b9
I20260809 00:32:48.438917 23959 ts_tablet_manager.cc:1939] T 22964da0834d441a8ecb127db927c1b9 P 96f72c86530f4c868f317a250a3cfe85: Deleting consensus metadata
I20260809 00:32:48.439308 23389 catalog_manager.cc:4841] TS 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194:45017): tablet 22964da0834d441a8ecb127db927c1b9 (table test-table [id=56acb07e34674869afd0d349033cc8c2]) successfully deleted
I20260809 00:32:48.442091 23960 tablet_replica.cc:330] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: stopping tablet replica
I20260809 00:32:48.442279 23960 raft_consensus.cc:2229] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:32:48.442386 23960 raft_consensus.cc:2258] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:48.442831 23960 ts_tablet_manager.cc:1905] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:48.444444 23960 ts_tablet_manager.cc:1918] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:32:48.444522 23960 log.cc:1196] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/wal/wals/22964da0834d441a8ecb127db927c1b9
I20260809 00:32:48.444857 23960 ts_tablet_manager.cc:1939] T 22964da0834d441a8ecb127db927c1b9 P 5aaec76af4d4476ab475c33124a301db: Deleting consensus metadata
I20260809 00:32:48.445286 23390 catalog_manager.cc:4841] TS 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947): tablet 22964da0834d441a8ecb127db927c1b9 (table test-table [id=56acb07e34674869afd0d349033cc8c2]) successfully deleted
I20260809 00:32:48.450402 23916 raft_consensus.cc:482] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:32:48.450479 23916 raft_consensus.cc:504] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.450794 23916 leader_election.cc:288] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947), 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194:45017)
I20260809 00:32:48.454735 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.450986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:36947 (local address 127.21.41.195:58583)
0809 00:32:48.451030 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.451044 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.451089 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.451532 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.451533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.451534 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:48.451601 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.451603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.452492 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.452493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.453667 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.453669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.454468 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.454469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.454551 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.454555 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.454603 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.454643 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 00:32:48.455343 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.451228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:58583 (local address 127.21.41.193:36947)
0809 00:32:48.451397 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.451399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.451407 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:32:48.451430 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.451431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.451439 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.451487 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:48.451668 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.452392 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.453775 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.454314 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.454436 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.454447 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.455230 (+ 783us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.455275 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260809 00:32:48.455405 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.451168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:34861 (local address 127.21.41.194:45017)
0809 00:32:48.451285 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.451287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.451297 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:32:48.451337 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.451337 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.451347 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.451400 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:32:48.452174 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.452928 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.454363 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.454911 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.454959 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.454966 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.455298 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.455336 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 00:32:48.455559 23561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302f9b6919bf4f6290ae50588fdea56d" candidate_uuid: "059874e6a7894fe5b503b7f826fb05b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aaec76af4d4476ab475c33124a301db" is_pre_election: true
I20260809 00:32:48.455637 23561 raft_consensus.cc:2454] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 059874e6a7894fe5b503b7f826fb05b9 in term 0.
I20260809 00:32:48.455804 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.450989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:45017 (local address 127.21.41.195:34861)
0809 00:32:48.451201 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.451212 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.451261 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.451686 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.451687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.451689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:32:48.452076 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.452078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.453031 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.453032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.454234 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.454236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.455112 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.455113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.455199 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.455203 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.455636 (+ 433us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.455721 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":13,"thread_start_us":59,"threads_started":1}
I20260809 00:32:48.455950 23774 leader_election.cc:302] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059874e6a7894fe5b503b7f826fb05b9, 5aaec76af4d4476ab475c33124a301db; no voters:
I20260809 00:32:48.456075 23916 raft_consensus.cc:2781] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:32:48.456102 23916 raft_consensus.cc:482] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:32:48.456121 23916 raft_consensus.cc:3037] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:48.456130 23701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302f9b6919bf4f6290ae50588fdea56d" candidate_uuid: "059874e6a7894fe5b503b7f826fb05b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f72c86530f4c868f317a250a3cfe85" is_pre_election: true
I20260809 00:32:48.456220 23701 raft_consensus.cc:2454] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 059874e6a7894fe5b503b7f826fb05b9 in term 0.
I20260809 00:32:48.457051 23916 raft_consensus.cc:504] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.457191 23916 leader_election.cc:288] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [CANDIDATE]: Term 1 election: Requested vote from peers 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947), 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194:45017)
I20260809 00:32:48.457365 23701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302f9b6919bf4f6290ae50588fdea56d" candidate_uuid: "059874e6a7894fe5b503b7f826fb05b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f72c86530f4c868f317a250a3cfe85"
I20260809 00:32:48.457389 23561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302f9b6919bf4f6290ae50588fdea56d" candidate_uuid: "059874e6a7894fe5b503b7f826fb05b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aaec76af4d4476ab475c33124a301db"
I20260809 00:32:48.457441 23701 raft_consensus.cc:3037] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:48.457453 23561 raft_consensus.cc:3037] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:48.458199 23561 raft_consensus.cc:2454] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 059874e6a7894fe5b503b7f826fb05b9 in term 1.
I20260809 00:32:48.458251 23701 raft_consensus.cc:2454] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 059874e6a7894fe5b503b7f826fb05b9 in term 1.
I20260809 00:32:48.458456 23774 leader_election.cc:302] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059874e6a7894fe5b503b7f826fb05b9, 5aaec76af4d4476ab475c33124a301db; no voters:
I20260809 00:32:48.458531 23916 raft_consensus.cc:2781] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:48.458660 23916 raft_consensus.cc:686] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 1 LEADER]: Becoming Leader. State: Replica: 059874e6a7894fe5b503b7f826fb05b9, State: Running, Role: LEADER
I20260809 00:32:48.458730 23916 consensus_queue.cc:237] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:48.459499 23402 catalog_manager.cc:5495] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195). New cstate: current_term: 1 leader_uuid: "059874e6a7894fe5b503b7f826fb05b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } health_report { overall_health: HEALTHY } } }
I20260809 00:32:48.473125 23906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.470028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.21.41.195:44027)
0809 00:32:48.470068 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.470070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.470078 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:32:48.470115 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.470115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.470138 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.470219 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:48.470384 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.471093 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.472209 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.472519 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.472594 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.472762 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.472889 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.472971 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.473018 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.473044 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:32:48.478883 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.475881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:53361 (local address 127.21.41.193:36947)
0809 00:32:48.475927 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.475929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.475938 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:32:48.475994 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.475995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.476007 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.476055 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:48.476206 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.476942 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.477964 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.478274 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.478393 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.478595 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.478630 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.478685 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.478778 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.478800 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:32:48.478899 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.475804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:36947 (local address 127.21.41.195:53361)
0809 00:32:48.475870 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.475881 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.475924 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.476080 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.476081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.476082 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:32:48.476139 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.476141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.477033 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.477034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.477868 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.477870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.478413 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.478414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.478484 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.478500 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.478703 (+ 203us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.478705 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.478773 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.478824 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 00:32:48.481045 23700 raft_consensus.cc:1261] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Refusing update from remote peer 059874e6a7894fe5b503b7f826fb05b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:48.481359 23965 consensus_queue.cc:1035] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:48.481573 23561 raft_consensus.cc:1261] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Refusing update from remote peer 059874e6a7894fe5b503b7f826fb05b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:48.481894 23965 consensus_queue.cc:1035] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:48.483268 23940 mvcc.cc:205] Tried to move back new op lower bound from 7316420888496103424 to 7316420888407429120. Current Snapshot: MvccSnapshot[applied={T|T < 7316420888496103424}]
I20260809 00:32:48.491106 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.488116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:44027 (local address 127.21.41.193:47125)
0809 00:32:48.488167 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:48.488178 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:48.488216 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:48.488404 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:48.488405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:48.488406 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:32:48.488449 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.488451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.489349 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.489350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.490217 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:48.490220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.490712 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.490713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:48.490768 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.490775 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.490943 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.490945 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:32:48.490983 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 00:32:48.491025 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 00:32:48.491496 23906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:48.488142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:47125 (local address 127.21.41.195:44027)
0809 00:32:48.488289 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:48.488291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:48.488299 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:32:48.488318 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:48.488319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:48.488328 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:48.488382 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:48.488532 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.489255 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.490308 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:48.490623 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:48.490679 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:48.490847 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:48.490881 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:48.490921 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:48.491413 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 00:32:48.491432 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
W20260809 00:32:48.491746 23940 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 00:32:50.017607 23402 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:50.017747 23402 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:50.018570 23402 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5: Sending DeleteTablet for 3 replicas of tablet 302f9b6919bf4f6290ae50588fdea56d
I20260809 00:32:50.018843 23681 tablet_service.cc:1514] Processing DeleteTablet for tablet 302f9b6919bf4f6290ae50588fdea56d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:50 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:55122
I20260809 00:32:50.018865 23541 tablet_service.cc:1514] Processing DeleteTablet for tablet 302f9b6919bf4f6290ae50588fdea56d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:50 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:54938
I20260809 00:32:50.019065 23995 tablet_replica.cc:330] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db: stopping tablet replica
I20260809 00:32:50.019093 23821 tablet_service.cc:1514] Processing DeleteTablet for tablet 302f9b6919bf4f6290ae50588fdea56d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:50 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:48218
I20260809 00:32:50.019127 23995 raft_consensus.cc:2229] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:32:50.019182 23995 raft_consensus.cc:2258] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:50.019397 23996 tablet_replica.cc:330] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9: stopping tablet replica
I20260809 00:32:50.019459 23995 ts_tablet_manager.cc:1905] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:50.019485 23996 raft_consensus.cc:2229] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:32:50.019613 23996 raft_consensus.cc:2258] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:50.019893 23996 ts_tablet_manager.cc:1905] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:50.021421 23995 ts_tablet_manager.cc:1918] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:32:50.021493 23995 log.cc:1196] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/wal/wals/302f9b6919bf4f6290ae50588fdea56d
I20260809 00:32:50.021735 23995 ts_tablet_manager.cc:1939] T 302f9b6919bf4f6290ae50588fdea56d P 5aaec76af4d4476ab475c33124a301db: Deleting consensus metadata
I20260809 00:32:50.021775 23996 ts_tablet_manager.cc:1918] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:32:50.021848 23996 log.cc:1196] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/302f9b6919bf4f6290ae50588fdea56d
I20260809 00:32:50.022084 23390 catalog_manager.cc:4841] TS 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947): tablet 302f9b6919bf4f6290ae50588fdea56d (table test-table [id=79113b44c7f84072b1ecbe2bd3906325]) successfully deleted
I20260809 00:32:50.022377 23996 ts_tablet_manager.cc:1939] T 302f9b6919bf4f6290ae50588fdea56d P 059874e6a7894fe5b503b7f826fb05b9: Deleting consensus metadata
I20260809 00:32:50.022509 23994 tablet_replica.cc:330] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85: stopping tablet replica
I20260809 00:32:50.022600 23994 raft_consensus.cc:2229] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:32:50.022643 23994 raft_consensus.cc:2258] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:50.022675 23389 catalog_manager.cc:4841] TS 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195:44027): tablet 302f9b6919bf4f6290ae50588fdea56d (table test-table [id=79113b44c7f84072b1ecbe2bd3906325]) successfully deleted
I20260809 00:32:50.023005 23994 ts_tablet_manager.cc:1905] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:50.024256 23994 ts_tablet_manager.cc:1918] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:32:50.024333 23994 log.cc:1196] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/302f9b6919bf4f6290ae50588fdea56d
I20260809 00:32:50.024538 23994 ts_tablet_manager.cc:1939] T 302f9b6919bf4f6290ae50588fdea56d P 96f72c86530f4c868f317a250a3cfe85: Deleting consensus metadata
I20260809 00:32:50.024863 23389 catalog_manager.cc:4841] TS 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194:45017): tablet 302f9b6919bf4f6290ae50588fdea56d (table test-table [id=79113b44c7f84072b1ecbe2bd3906325]) successfully deleted
Aug 09 00:32:50 dist-test-slave-zpfg krb5kdc[23359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235570, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:32:50 dist-test-slave-zpfg krb5kdc[23359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235570, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:50.044811 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:50.036560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48982 (local address 127.21.41.254:33743)
0809 00:32:50.036721 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:50.036723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:50.036735 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:32:50.036822 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:50.036822 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:50.036835 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:50.036920 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:50.037528 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:50.038059 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:50.038865 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:50.039075 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:50.039166 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:50.042046 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:50.042047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:50.042048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:50.042056 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:50.043625 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:50.043971 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:50.043972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:50.043972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:50.044035 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:50.044241 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:50.044242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:50.044242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:50.044355 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:50.044495 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:50.044609 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 00:32:50.044698 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260809 00:32:50.046270 23402 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:32:50.046478 23402 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:32:50.047830 23541 tablet_service.cc:1467] Processing CreateTablet for tablet cd0235e0c14a4b96a41305a9b8d9ae36 (DEFAULT_TABLE table=test-table [id=d76303af1c2f40f5a35a1af73f357c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:50.047996 23541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd0235e0c14a4b96a41305a9b8d9ae36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:50.049265 24005 tablet_bootstrap.cc:492] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db: Bootstrap starting.
I20260809 00:32:50.049319 23681 tablet_service.cc:1467] Processing CreateTablet for tablet cd0235e0c14a4b96a41305a9b8d9ae36 (DEFAULT_TABLE table=test-table [id=d76303af1c2f40f5a35a1af73f357c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:50.049444 23681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd0235e0c14a4b96a41305a9b8d9ae36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:50.049705 24005 tablet_bootstrap.cc:654] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:50.050418 24005 tablet_bootstrap.cc:492] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db: No bootstrap required, opened a new log
I20260809 00:32:50.050478 24005 ts_tablet_manager.cc:1397] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:50.050491 23821 tablet_service.cc:1467] Processing CreateTablet for tablet cd0235e0c14a4b96a41305a9b8d9ae36 (DEFAULT_TABLE table=test-table [id=d76303af1c2f40f5a35a1af73f357c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:50.050621 23821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd0235e0c14a4b96a41305a9b8d9ae36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:50.050581 24005 raft_consensus.cc:348] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.050650 24005 raft_consensus.cc:374] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:50.050670 24005 raft_consensus.cc:729] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aaec76af4d4476ab475c33124a301db, State: Initialized, Role: FOLLOWER
I20260809 00:32:50.050743 24005 consensus_queue.cc:260] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.051038 24005 ts_tablet_manager.cc:1428] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:32:50.051285 24007 tablet_bootstrap.cc:492] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85: Bootstrap starting.
I20260809 00:32:50.051736 24007 tablet_bootstrap.cc:654] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:50.052105 24009 tablet_bootstrap.cc:492] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9: Bootstrap starting.
I20260809 00:32:50.052294 24007 tablet_bootstrap.cc:492] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85: No bootstrap required, opened a new log
I20260809 00:32:50.052333 24007 ts_tablet_manager.cc:1397] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:50.052414 24007 raft_consensus.cc:348] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.052465 24007 raft_consensus.cc:374] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:50.052479 24007 raft_consensus.cc:729] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f72c86530f4c868f317a250a3cfe85, State: Initialized, Role: FOLLOWER
I20260809 00:32:50.052538 24007 consensus_queue.cc:260] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.052700 24007 ts_tablet_manager.cc:1428] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:50.053170 24009 tablet_bootstrap.cc:654] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:50.053735 24009 tablet_bootstrap.cc:492] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9: No bootstrap required, opened a new log
I20260809 00:32:50.053785 24009 ts_tablet_manager.cc:1397] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:32:50.053869 24009 raft_consensus.cc:348] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.053920 24009 raft_consensus.cc:374] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:50.053934 24009 raft_consensus.cc:729] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059874e6a7894fe5b503b7f826fb05b9, State: Initialized, Role: FOLLOWER
I20260809 00:32:50.053994 24009 consensus_queue.cc:260] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.054095 24009 ts_tablet_manager.cc:1428] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:50.178433 23986 raft_consensus.cc:482] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:32:50.178563 23986 raft_consensus.cc:504] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.178805 23986 leader_election.cc:288] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947), 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194:45017)
I20260809 00:32:50.179061 23700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd0235e0c14a4b96a41305a9b8d9ae36" candidate_uuid: "059874e6a7894fe5b503b7f826fb05b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f72c86530f4c868f317a250a3cfe85" is_pre_election: true
I20260809 00:32:50.179054 23561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd0235e0c14a4b96a41305a9b8d9ae36" candidate_uuid: "059874e6a7894fe5b503b7f826fb05b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aaec76af4d4476ab475c33124a301db" is_pre_election: true
I20260809 00:32:50.179168 23561 raft_consensus.cc:2454] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 059874e6a7894fe5b503b7f826fb05b9 in term 0.
I20260809 00:32:50.179168 23700 raft_consensus.cc:2454] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 059874e6a7894fe5b503b7f826fb05b9 in term 0.
I20260809 00:32:50.179345 23773 leader_election.cc:302] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059874e6a7894fe5b503b7f826fb05b9, 96f72c86530f4c868f317a250a3cfe85; no voters:
I20260809 00:32:50.179474 23986 raft_consensus.cc:2781] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:32:50.179527 23986 raft_consensus.cc:482] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:32:50.179557 23986 raft_consensus.cc:3037] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:50.180297 23986 raft_consensus.cc:504] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.180444 23986 leader_election.cc:288] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [CANDIDATE]: Term 1 election: Requested vote from peers 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947), 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194:45017)
I20260809 00:32:50.180624 23700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd0235e0c14a4b96a41305a9b8d9ae36" candidate_uuid: "059874e6a7894fe5b503b7f826fb05b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f72c86530f4c868f317a250a3cfe85"
I20260809 00:32:50.180636 23561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd0235e0c14a4b96a41305a9b8d9ae36" candidate_uuid: "059874e6a7894fe5b503b7f826fb05b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aaec76af4d4476ab475c33124a301db"
I20260809 00:32:50.180721 23561 raft_consensus.cc:3037] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:50.180727 23700 raft_consensus.cc:3037] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:50.181344 23561 raft_consensus.cc:2454] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 059874e6a7894fe5b503b7f826fb05b9 in term 1.
I20260809 00:32:50.181572 23700 raft_consensus.cc:2454] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 059874e6a7894fe5b503b7f826fb05b9 in term 1.
I20260809 00:32:50.181591 23774 leader_election.cc:302] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059874e6a7894fe5b503b7f826fb05b9, 5aaec76af4d4476ab475c33124a301db; no voters:
I20260809 00:32:50.181689 23986 raft_consensus.cc:2781] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:50.181757 23986 raft_consensus.cc:686] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 LEADER]: Becoming Leader. State: Replica: 059874e6a7894fe5b503b7f826fb05b9, State: Running, Role: LEADER
I20260809 00:32:50.181825 23986 consensus_queue.cc:237] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.182595 23402 catalog_manager.cc:5495] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195). New cstate: current_term: 1 leader_uuid: "059874e6a7894fe5b503b7f826fb05b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } health_report { overall_health: HEALTHY } } }
I20260809 00:32:50.199436 24014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:50.196912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.21.41.195:44027)
0809 00:32:50.197097 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:50.197098 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:50.197113 (+ 15us) server_negotiation.cc:407] Connection header received
0809 00:32:50.197131 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:50.197132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:50.197141 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:50.197199 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:50.197358 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:50.197829 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:50.198705 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:50.198904 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:50.198971 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:50.199159 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:50.199189 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:50.199235 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:50.199288 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 00:32:50.199308 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":120,"threads_started":1}
I20260809 00:32:50.201941 23561 raft_consensus.cc:1261] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Refusing update from remote peer 059874e6a7894fe5b503b7f826fb05b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:50.201951 23700 raft_consensus.cc:1261] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Refusing update from remote peer 059874e6a7894fe5b503b7f826fb05b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:50.202185 23972 consensus_queue.cc:1035] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:50.202287 23986 consensus_queue.cc:1035] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:50.212265 23403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:50.212376 23403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:50.213122 23403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5: Sending DeleteTablet for 3 replicas of tablet cd0235e0c14a4b96a41305a9b8d9ae36
I20260809 00:32:50.213384 23541 tablet_service.cc:1514] Processing DeleteTablet for tablet cd0235e0c14a4b96a41305a9b8d9ae36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:50 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:54938
I20260809 00:32:50.213426 23821 tablet_service.cc:1514] Processing DeleteTablet for tablet cd0235e0c14a4b96a41305a9b8d9ae36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:50 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:48218
I20260809 00:32:50.213409 23681 tablet_service.cc:1514] Processing DeleteTablet for tablet cd0235e0c14a4b96a41305a9b8d9ae36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:50 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:55122
I20260809 00:32:50.213491 23995 tablet_replica.cc:330] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db: stopping tablet replica
I20260809 00:32:50.213539 23996 tablet_replica.cc:330] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9: stopping tablet replica
I20260809 00:32:50.213550 23995 raft_consensus.cc:2229] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:32:50.213549 23994 tablet_replica.cc:330] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85: stopping tablet replica
I20260809 00:32:50.213591 23995 raft_consensus.cc:2258] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:50.213594 23996 raft_consensus.cc:2229] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:32:50.213609 23994 raft_consensus.cc:2229] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:32:50.213646 23996 raft_consensus.cc:2258] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:50.213653 23994 raft_consensus.cc:2258] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:50.213840 23403 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 00:32:50.213934 23996 ts_tablet_manager.cc:1905] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:32:50.214010 23403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:32:50.214102 23995 ts_tablet_manager.cc:1905] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:50.214407 23994 ts_tablet_manager.cc:1905] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:50.215232 23541 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd2f03148f24c88bd32208ffe38d702 (DEFAULT_TABLE table=test-table [id=846f0690b88e47b9bac8a5289957499d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:50.215339 23541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd2f03148f24c88bd32208ffe38d702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:50.215801 23996 ts_tablet_manager.cc:1918] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:32:50.215842 23996 log.cc:1196] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/cd0235e0c14a4b96a41305a9b8d9ae36
I20260809 00:32:50.216009 23996 ts_tablet_manager.cc:1939] T cd0235e0c14a4b96a41305a9b8d9ae36 P 059874e6a7894fe5b503b7f826fb05b9: Deleting consensus metadata
I20260809 00:32:50.216055 23994 ts_tablet_manager.cc:1918] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:32:50.216111 23994 log.cc:1196] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/cd0235e0c14a4b96a41305a9b8d9ae36
I20260809 00:32:50.216244 23681 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd2f03148f24c88bd32208ffe38d702 (DEFAULT_TABLE table=test-table [id=846f0690b88e47b9bac8a5289957499d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:50.216341 23681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd2f03148f24c88bd32208ffe38d702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:50.216365 23994 ts_tablet_manager.cc:1939] T cd0235e0c14a4b96a41305a9b8d9ae36 P 96f72c86530f4c868f317a250a3cfe85: Deleting consensus metadata
I20260809 00:32:50.216697 23995 ts_tablet_manager.cc:1918] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:32:50.216724 23389 catalog_manager.cc:4841] TS 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194:45017): tablet cd0235e0c14a4b96a41305a9b8d9ae36 (table test-table [id=d76303af1c2f40f5a35a1af73f357c3c]) successfully deleted
I20260809 00:32:50.216749 23995 log.cc:1196] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-0/wal/wals/cd0235e0c14a4b96a41305a9b8d9ae36
I20260809 00:32:50.216910 23389 catalog_manager.cc:4841] TS 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195:44027): tablet cd0235e0c14a4b96a41305a9b8d9ae36 (table test-table [id=d76303af1c2f40f5a35a1af73f357c3c]) successfully deleted
I20260809 00:32:50.216984 23995 ts_tablet_manager.cc:1939] T cd0235e0c14a4b96a41305a9b8d9ae36 P 5aaec76af4d4476ab475c33124a301db: Deleting consensus metadata
I20260809 00:32:50.216969 23821 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd2f03148f24c88bd32208ffe38d702 (DEFAULT_TABLE table=test-table [id=846f0690b88e47b9bac8a5289957499d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:32:50.217061 23821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd2f03148f24c88bd32208ffe38d702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:50.217419 24007 tablet_bootstrap.cc:492] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85: Bootstrap starting.
I20260809 00:32:50.217754 24007 tablet_bootstrap.cc:654] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:50.218075 23390 catalog_manager.cc:4841] TS 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947): tablet cd0235e0c14a4b96a41305a9b8d9ae36 (table test-table [id=d76303af1c2f40f5a35a1af73f357c3c]) successfully deleted
I20260809 00:32:50.219063 24005 tablet_bootstrap.cc:492] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db: Bootstrap starting.
I20260809 00:32:50.219101 24009 tablet_bootstrap.cc:492] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9: Bootstrap starting.
I20260809 00:32:50.219422 24005 tablet_bootstrap.cc:654] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:50.219457 24009 tablet_bootstrap.cc:654] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:50.219892 24007 tablet_bootstrap.cc:492] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85: No bootstrap required, opened a new log
I20260809 00:32:50.219980 24007 ts_tablet_manager.cc:1397] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:32:50.220059 24005 tablet_bootstrap.cc:492] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db: No bootstrap required, opened a new log
I20260809 00:32:50.220088 24007 raft_consensus.cc:348] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.220129 24005 ts_tablet_manager.cc:1397] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:50.220151 24007 raft_consensus.cc:374] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:50.220166 24007 raft_consensus.cc:729] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f72c86530f4c868f317a250a3cfe85, State: Initialized, Role: FOLLOWER
I20260809 00:32:50.220232 24007 consensus_queue.cc:260] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.220337 24007 ts_tablet_manager.cc:1428] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:50.220299 24005 raft_consensus.cc:348] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.220371 24005 raft_consensus.cc:374] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:50.220399 24005 raft_consensus.cc:729] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aaec76af4d4476ab475c33124a301db, State: Initialized, Role: FOLLOWER
I20260809 00:32:50.220440 24005 consensus_queue.cc:260] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.220541 24005 ts_tablet_manager.cc:1428] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:50.220556 24009 tablet_bootstrap.cc:492] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9: No bootstrap required, opened a new log
I20260809 00:32:50.220598 24009 ts_tablet_manager.cc:1397] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:32:50.220710 24009 raft_consensus.cc:348] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.220764 24009 raft_consensus.cc:374] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:50.220800 24009 raft_consensus.cc:729] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059874e6a7894fe5b503b7f826fb05b9, State: Initialized, Role: FOLLOWER
I20260809 00:32:50.220855 24009 consensus_queue.cc:260] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.220955 24009 ts_tablet_manager.cc:1428] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:50.222385 24010 raft_consensus.cc:482] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:32:50.222450 24010 raft_consensus.cc:504] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.222549 24010 leader_election.cc:288] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947), 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195:44027)
I20260809 00:32:50.222725 23561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd2f03148f24c88bd32208ffe38d702" candidate_uuid: "96f72c86530f4c868f317a250a3cfe85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aaec76af4d4476ab475c33124a301db" is_pre_election: true
I20260809 00:32:50.222790 23561 raft_consensus.cc:2454] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f72c86530f4c868f317a250a3cfe85 in term 0.
I20260809 00:32:50.222891 23841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd2f03148f24c88bd32208ffe38d702" candidate_uuid: "96f72c86530f4c868f317a250a3cfe85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "059874e6a7894fe5b503b7f826fb05b9" is_pre_election: true
I20260809 00:32:50.222927 23634 leader_election.cc:302] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aaec76af4d4476ab475c33124a301db, 96f72c86530f4c868f317a250a3cfe85; no voters:
I20260809 00:32:50.222971 23841 raft_consensus.cc:2454] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f72c86530f4c868f317a250a3cfe85 in term 0.
I20260809 00:32:50.223048 24010 raft_consensus.cc:2781] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:32:50.223095 24010 raft_consensus.cc:482] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:32:50.223124 24010 raft_consensus.cc:3037] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:50.223693 24010 raft_consensus.cc:504] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.223825 24010 leader_election.cc:288] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [CANDIDATE]: Term 1 election: Requested vote from peers 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947), 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195:44027)
I20260809 00:32:50.223956 23841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd2f03148f24c88bd32208ffe38d702" candidate_uuid: "96f72c86530f4c868f317a250a3cfe85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "059874e6a7894fe5b503b7f826fb05b9"
I20260809 00:32:50.223951 23561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd2f03148f24c88bd32208ffe38d702" candidate_uuid: "96f72c86530f4c868f317a250a3cfe85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aaec76af4d4476ab475c33124a301db"
I20260809 00:32:50.224018 23841 raft_consensus.cc:3037] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:50.224018 23561 raft_consensus.cc:3037] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:50.224705 23561 raft_consensus.cc:2454] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f72c86530f4c868f317a250a3cfe85 in term 1.
I20260809 00:32:50.224732 23841 raft_consensus.cc:2454] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f72c86530f4c868f317a250a3cfe85 in term 1.
I20260809 00:32:50.224874 23633 leader_election.cc:302] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059874e6a7894fe5b503b7f826fb05b9, 96f72c86530f4c868f317a250a3cfe85; no voters:
I20260809 00:32:50.224994 24010 raft_consensus.cc:2781] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:50.225121 24010 raft_consensus.cc:686] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 1 LEADER]: Becoming Leader. State: Replica: 96f72c86530f4c868f317a250a3cfe85, State: Running, Role: LEADER
I20260809 00:32:50.225193 24010 consensus_queue.cc:237] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } }
I20260809 00:32:50.225724 23402 catalog_manager.cc:5495] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194). New cstate: current_term: 1 leader_uuid: "96f72c86530f4c868f317a250a3cfe85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f72c86530f4c868f317a250a3cfe85" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 } health_report { overall_health: UNKNOWN } } }
I20260809 00:32:50.233850 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:50.231526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55138 (local address 127.21.41.194:45017)
0809 00:32:50.231717 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:50.231718 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:50.231724 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:32:50.231741 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:50.231741 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:50.231751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:50.231826 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:32:50.231940 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:50.232420 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:50.233135 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:50.233350 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:50.233435 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:50.233581 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:32:50.233621 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:32:50.233683 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:50.233744 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 00:32:50.233766 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":121,"threads_started":1}
I20260809 00:32:50.236019 23561 raft_consensus.cc:1261] T 0bd2f03148f24c88bd32208ffe38d702 P 5aaec76af4d4476ab475c33124a301db [term 1 FOLLOWER]: Refusing update from remote peer 96f72c86530f4c868f317a250a3cfe85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:50.236019 23841 raft_consensus.cc:1261] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Refusing update from remote peer 96f72c86530f4c868f317a250a3cfe85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:32:50.236272 24010 consensus_queue.cc:1035] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "059874e6a7894fe5b503b7f826fb05b9" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:32:50.236370 24022 consensus_queue.cc:1035] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaec76af4d4476ab475c33124a301db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:32:50.240653 23978 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 00:32:51.757875 23402 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:51.758035 23402 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:32:51.758894 23402 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee0b8e4020d44b59938f8ab4f47282c5: Sending DeleteTablet for 3 replicas of tablet 0bd2f03148f24c88bd32208ffe38d702
I20260809 00:32:51.759156 23541 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bd2f03148f24c88bd32208ffe38d702 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:51 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:54938
I20260809 00:32:51.759204 23681 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bd2f03148f24c88bd32208ffe38d702 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:51 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:55122
I20260809 00:32:51.759204 23821 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bd2f03148f24c88bd32208ffe38d702 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:32:51 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:48218
I20260809 00:32:51.759327 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 23476
I20260809 00:32:51.760126 24052 tablet_replica.cc:330] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9: stopping tablet replica
I20260809 00:32:51.760200 24052 raft_consensus.cc:2229] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:32:51.760246 24052 raft_consensus.cc:2258] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:51.760555 24052 ts_tablet_manager.cc:1905] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:51.760996 24053 tablet_replica.cc:330] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85: stopping tablet replica
I20260809 00:32:51.761067 24053 raft_consensus.cc:2229] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:32:51.761133 24053 raft_consensus.cc:2258] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:32:51.761379 24053 ts_tablet_manager.cc:1905] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:32:51.762657 24052 ts_tablet_manager.cc:1918] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:32:51.762722 24052 log.cc:1196] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/0bd2f03148f24c88bd32208ffe38d702
I20260809 00:32:51.762943 24052 ts_tablet_manager.cc:1939] T 0bd2f03148f24c88bd32208ffe38d702 P 059874e6a7894fe5b503b7f826fb05b9: Deleting consensus metadata
I20260809 00:32:51.763172 24053 ts_tablet_manager.cc:1918] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:32:51.763231 24053 log.cc:1196] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/0bd2f03148f24c88bd32208ffe38d702
I20260809 00:32:51.763389 23389 catalog_manager.cc:4841] TS 059874e6a7894fe5b503b7f826fb05b9 (127.21.41.195:44027): tablet 0bd2f03148f24c88bd32208ffe38d702 (table test-table [id=846f0690b88e47b9bac8a5289957499d]) successfully deleted
I20260809 00:32:51.763444 24053 ts_tablet_manager.cc:1939] T 0bd2f03148f24c88bd32208ffe38d702 P 96f72c86530f4c868f317a250a3cfe85: Deleting consensus metadata
I20260809 00:32:51.763764 23389 catalog_manager.cc:4841] TS 96f72c86530f4c868f317a250a3cfe85 (127.21.41.194:45017): tablet 0bd2f03148f24c88bd32208ffe38d702 (table test-table [id=846f0690b88e47b9bac8a5289957499d]) successfully deleted
W20260809 00:32:51.767112 23390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:36947 (error 108)
I20260809 00:32:51.767370 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 23616
I20260809 00:32:51.768082 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:51.767551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:36947 (local address 127.0.0.1:54966)
0809 00:32:51.767992 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:51.768039 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:36947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":69,"threads_started":1}
W20260809 00:32:51.768234 23390 catalog_manager.cc:4584] TS 5aaec76af4d4476ab475c33124a301db (127.21.41.193:36947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0bd2f03148f24c88bd32208ffe38d702: Network error: Client connection negotiation failed: client connection to 127.21.41.193:36947: connect: Connection refused (error 111)
I20260809 00:32:51.774575 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 23756
I20260809 00:32:51.780653 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 23372
2026-08-09T00:32:51Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5498 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:32:51 dist-test-slave-zpfg krb5kdc[24058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:32:51 dist-test-slave-zpfg krb5kdc[24058](info): set up 2 sockets
Aug 09 00:32:51 dist-test-slave-zpfg krb5kdc[24058](info): commencing operation
krb5kdc: starting...
W20260809 00:32:52.835723 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:32:52 dist-test-slave-zpfg krb5kdc[24058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:32:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:32:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:32:52.959223 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:40193
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:40193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:53.043849 24071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:53.044091 24071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:53.044127 24071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:53.045271 24071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:32:53.045341 24071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:53.045367 24071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:53.045397 24071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:32:53.045445 24071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:32:53.047001 24071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:40193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:40193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:53.047281 24071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:53.047518 24071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:53.053247 24076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:53.053467 24071 server_base.cc:1034] running on GCE node
W20260809 00:32:53.053660 24077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:53.053814 24079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:53.054091 24071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:53.054488 24071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:53.055742 24071 hybrid_clock.cc:648] HybridClock initialized: now 1786235573055687 us; error 68 us; skew 500 ppm
Aug 09 00:32:53 dist-test-slave-zpfg krb5kdc[24058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235573, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:53.058537 24071 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:32:53.059028 24071 webserver.cc:460] Webserver started at http://127.21.41.254:42693/ using document root <none> and password file <none>
I20260809 00:32:53.059305 24071 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:53.059376 24071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:53.059530 24071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:53.060631 24071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "12598fdd49fd4078a3a1d1f86aaa6d33"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "a405c16e05b2d21142dab47efb350d95"
server_key_iv: "ead013778d76feea33adf2bbbe4c926b"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.061002 24071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "12598fdd49fd4078a3a1d1f86aaa6d33"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "a405c16e05b2d21142dab47efb350d95"
server_key_iv: "ead013778d76feea33adf2bbbe4c926b"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.062330 24071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:53.063339 24086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:53.063539 24071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:32:53.063625 24071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "12598fdd49fd4078a3a1d1f86aaa6d33"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "a405c16e05b2d21142dab47efb350d95"
server_key_iv: "ead013778d76feea33adf2bbbe4c926b"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.063674 24071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:53.079468 24071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:53.082587 24071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:53.082754 24071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:53.087733 24071 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:40193
I20260809 00:32:53.087867 24138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:40193 every 8 connection(s)
I20260809 00:32:53.088199 24071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:32:53.089365 24139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:53.091979 24139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33: Bootstrap starting.
I20260809 00:32:53.092549 24139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:53.092813 24139 log.cc:824] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:53.093504 24139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33: No bootstrap required, opened a new log
I20260809 00:32:53.094671 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 24071
I20260809 00:32:53.094774 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:32:53.094959 21671 external_mini_cluster.cc:1428] Setting key 8e2feb442f98f83b68f09e54d11f27bf
I20260809 00:32:53.095976 24139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12598fdd49fd4078a3a1d1f86aaa6d33" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40193 } }
I20260809 00:32:53.096176 24139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:53.096230 24139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12598fdd49fd4078a3a1d1f86aaa6d33, State: Initialized, Role: FOLLOWER
I20260809 00:32:53.096385 24139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12598fdd49fd4078a3a1d1f86aaa6d33" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40193 } }
I20260809 00:32:53.096486 24139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:32:53.096524 24139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:32:53.096583 24139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:53.097796 24139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12598fdd49fd4078a3a1d1f86aaa6d33" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40193 } }
I20260809 00:32:53.097924 24139 leader_election.cc:302] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12598fdd49fd4078a3a1d1f86aaa6d33; no voters:
I20260809 00:32:53.098109 24139 leader_election.cc:288] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:32:53.098160 24144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:53.098438 24139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:32:53.098477 24144 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [term 1 LEADER]: Becoming Leader. State: Replica: 12598fdd49fd4078a3a1d1f86aaa6d33, State: Running, Role: LEADER
I20260809 00:32:53.098740 24144 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12598fdd49fd4078a3a1d1f86aaa6d33" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40193 } }
I20260809 00:32:53.099900 24146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12598fdd49fd4078a3a1d1f86aaa6d33. Latest consensus state: current_term: 1 leader_uuid: "12598fdd49fd4078a3a1d1f86aaa6d33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12598fdd49fd4078a3a1d1f86aaa6d33" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40193 } } }
I20260809 00:32:53.100005 24146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:53.100256 24145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12598fdd49fd4078a3a1d1f86aaa6d33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12598fdd49fd4078a3a1d1f86aaa6d33" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40193 } } }
I20260809 00:32:53.100473 24145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33 [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:53.100600 24160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:32:53.101135 24160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 00:32:53 dist-test-slave-zpfg krb5kdc[24058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:53.103134 24160 catalog_manager.cc:1347] Generated new cluster ID: 3dfe8f732a794f6f80474f6079e518ae
I20260809 00:32:53.103190 24160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:32:53.107731 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:53.095526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58198 (local address 127.21.41.254:40193)
0809 00:32:53.095725 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:53.095732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:53.095761 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:32:53.095929 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:53.095935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:53.095979 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:53.096088 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:53.096977 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.098065 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.099381 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.100099 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.100438 (+ 339us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:53.103576 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:53.103580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:53.103580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:53.103593 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:53.105608 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:53.106026 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:53.106030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:53.106030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:53.106164 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:53.106423 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:53.106424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:53.106424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:53.106746 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:53.106927 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:53.107363 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 00:32:53.107503 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260809 00:32:53.110644 24160 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:32:53.111208 24160 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:32:53.124114 24160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12598fdd49fd4078a3a1d1f86aaa6d33: Generated new TSK 0
I20260809 00:32:53.124346 24160 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:32:53.151158 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40193
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:53.234709 24166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:53.234915 24166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:53.234939 24166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:53.236173 24166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:53.236214 24166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:53.236240 24166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:32:53.237627 24166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:53.237812 24166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:53.237998 24166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:53.243188 24174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:53.243283 24172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:53.243188 24171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:53.243372 24166 server_base.cc:1034] running on GCE node
I20260809 00:32:53.243609 24166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:53.243810 24166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:53.244980 24166 hybrid_clock.cc:648] HybridClock initialized: now 1786235573244972 us; error 38 us; skew 500 ppm
Aug 09 00:32:53 dist-test-slave-zpfg krb5kdc[24058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235573, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:53.247699 24166 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:32:53.248198 24166 webserver.cc:460] Webserver started at http://127.21.41.193:44843/ using document root <none> and password file <none>
I20260809 00:32:53.248445 24166 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:53.248513 24166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:53.248661 24166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:53.249730 24166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "7d94320476ca40a6aba97661e59055fe"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "daa83585b6fafdb53516571a5c572ed4"
server_key_iv: "cb3ccb995ada609b709141d7a98af2b7"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.250079 24166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "7d94320476ca40a6aba97661e59055fe"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "daa83585b6fafdb53516571a5c572ed4"
server_key_iv: "cb3ccb995ada609b709141d7a98af2b7"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.251525 24166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:53.252326 24181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:53.252518 24166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:53.252616 24166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "7d94320476ca40a6aba97661e59055fe"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "daa83585b6fafdb53516571a5c572ed4"
server_key_iv: "cb3ccb995ada609b709141d7a98af2b7"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.252704 24166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:53.263502 24166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:53.265898 24166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:53.266070 24166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:53.266340 24166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:53.266729 24166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:53.266786 24166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:53.266826 24166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:53.266852 24166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:53.273183 24166 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:40035
I20260809 00:32:53.273236 24294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:40035 every 8 connection(s)
I20260809 00:32:53.273558 24166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:32:53.276520 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 24166
I20260809 00:32:53.276638 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:32:53.276818 21671 external_mini_cluster.cc:1428] Setting key f0821faf9cd0d79f1f3c7d30767d04fe
Aug 09 00:32:53 dist-test-slave-zpfg krb5kdc[24058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235573, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:53.284269 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:53.274285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:58331 (local address 127.21.41.254:40193)
0809 00:32:53.274330 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:53.274332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:53.274568 (+ 236us) server_negotiation.cc:407] Connection header received
0809 00:32:53.275163 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:53.275168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:53.275217 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:53.275301 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:53.276270 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.276865 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.278009 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.278270 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.278402 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:53.280910 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:53.280912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:53.280912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:53.280923 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:53.282759 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:53.283181 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:53.283182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:53.283183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:53.283257 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:53.283515 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:53.283516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:53.283517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:53.283671 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:53.283748 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:53.284038 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 00:32:53.284138 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:32:53.284302 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:53.274254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:40193 (local address 127.21.41.193:58331)
0809 00:32:53.274424 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:53.274471 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:53.275035 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:53.275379 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:53.275383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:53.275968 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:53.276161 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:53.276166 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.276960 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.276961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:53.277922 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:53.277924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.278409 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.278411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:53.278553 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:53.279248 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:53.279258 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:53.280809 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:53.282859 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:53.282866 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:53.282868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:53.283091 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:53.283340 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:53.283340 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:53.283341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:53.283445 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:53.283768 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:53.283776 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:53.283897 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 00:32:53.283992 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 00:32:53.284117 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":121,"threads_started":1}
I20260809 00:32:53.284806 24295 heartbeater.cc:347] Connected to a master server at 127.21.41.254:40193
I20260809 00:32:53.284910 24295 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:53.285116 24295 heartbeater.cc:511] Master 127.21.41.254:40193 requested a full tablet report, sending...
I20260809 00:32:53.285647 24103 ts_manager.cc:194] Registered new tserver with Master: 7d94320476ca40a6aba97661e59055fe (127.21.41.193:40035)
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
I20260809 00:32:53.286293 24103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:58331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:32:53.303575 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40193
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:53.385140 24300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:53.385355 24300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:53.385380 24300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:53.386615 24300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:53.386660 24300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:53.386687 24300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:32:53.388188 24300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:53.388446 24300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:53.388684 24300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:53.393833 24308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:53.393860 24305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:53.394085 24306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:53.394089 24300 server_base.cc:1034] running on GCE node
I20260809 00:32:53.394397 24300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:53.394618 24300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:53.395784 24300 hybrid_clock.cc:648] HybridClock initialized: now 1786235573395765 us; error 32 us; skew 500 ppm
Aug 09 00:32:53 dist-test-slave-zpfg krb5kdc[24058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235573, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:53.398588 24300 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:32:53.399142 24300 webserver.cc:460] Webserver started at http://127.21.41.194:34801/ using document root <none> and password file <none>
I20260809 00:32:53.399384 24300 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:53.399447 24300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:53.399580 24300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:53.400683 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "a59abdaf2238418a83aed082b653c570"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "a416dc0e19186de50acc27aa37012361"
server_key_iv: "ef2ce951e89be2b9cccdf72e5b6954a3"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.401047 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "a59abdaf2238418a83aed082b653c570"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "a416dc0e19186de50acc27aa37012361"
server_key_iv: "ef2ce951e89be2b9cccdf72e5b6954a3"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.402464 24300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:32:53.403237 24315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:53.403405 24300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:32:53.403501 24300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "a59abdaf2238418a83aed082b653c570"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "a416dc0e19186de50acc27aa37012361"
server_key_iv: "ef2ce951e89be2b9cccdf72e5b6954a3"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.403581 24300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:53.414175 24300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:53.416128 24300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:53.416298 24300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:53.416543 24300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:53.416859 24300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:53.416911 24300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:53.416953 24300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:53.416980 24300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:53.423311 24300 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:38957
I20260809 00:32:53.423362 24428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:38957 every 8 connection(s)
I20260809 00:32:53.423717 24300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:32:53.428844 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 24300
I20260809 00:32:53.428958 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:32:53.429150 21671 external_mini_cluster.cc:1428] Setting key 8e3cf624333247cf20e60d801d2b094b
Aug 09 00:32:53 dist-test-slave-zpfg krb5kdc[24058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235573, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:53.434477 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:53.424491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:44563 (local address 127.21.41.254:40193)
0809 00:32:53.424546 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:53.424549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:53.424748 (+ 199us) server_negotiation.cc:407] Connection header received
0809 00:32:53.425301 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:53.425303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:53.425316 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:53.425369 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:53.426417 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.426936 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.427990 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.428204 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.428281 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:53.431050 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:53.431052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:53.431052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:53.431061 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:53.432708 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:53.433310 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:53.433311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:53.433312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:53.433365 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:53.433674 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:53.433675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:53.433675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:53.433795 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:53.433847 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:53.434251 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 00:32:53.434325 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:32:53.434828 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:53.424440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:40193 (local address 127.21.41.194:44563)
0809 00:32:53.424600 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:53.424650 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:53.425169 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:53.425450 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:53.425455 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:53.426068 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:53.426275 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:53.426280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.427040 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.427041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:53.427888 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:53.427890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.428290 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.428291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:53.428421 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:53.429051 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:53.429061 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:53.430948 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:53.432818 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:53.432826 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:53.432829 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:53.433222 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:53.433433 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:53.433434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:53.433434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:53.433603 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:53.433864 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:53.433870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:53.434083 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 00:32:53.434236 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 00:32:53.434479 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260809 00:32:53.435437 24429 heartbeater.cc:347] Connected to a master server at 127.21.41.254:40193
I20260809 00:32:53.435587 24429 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:53.435833 24429 heartbeater.cc:511] Master 127.21.41.254:40193 requested a full tablet report, sending...
I20260809 00:32:53.436266 24103 ts_manager.cc:194] Registered new tserver with Master: a59abdaf2238418a83aed082b653c570 (127.21.41.194:38957)
I20260809 00:32:53.436658 24103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:44563
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:32:53.458190 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40193
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:53.544298 24434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:53.544514 24434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:53.544538 24434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:53.545789 24434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:53.545840 24434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:53.545898 24434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:32:53.547380 24434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:53.547646 24434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:53.547897 24434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:53.553205 24440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:53.553241 24442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:53.553231 24439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:53.553449 24434 server_base.cc:1034] running on GCE node
I20260809 00:32:53.553624 24434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:53.553825 24434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:53.554986 24434 hybrid_clock.cc:648] HybridClock initialized: now 1786235573554972 us; error 27 us; skew 500 ppm
Aug 09 00:32:53 dist-test-slave-zpfg krb5kdc[24058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235573, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:53.557909 24434 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:32:53.558492 24434 webserver.cc:460] Webserver started at http://127.21.41.195:46123/ using document root <none> and password file <none>
I20260809 00:32:53.558727 24434 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:53.558779 24434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:53.558940 24434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:53.560102 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "e7b0306d88784fe681642fa200f40f8d"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "537a009b7a9269c9bcee6dd90e78966c"
server_key_iv: "4404e934313df2a56629cadd03fd5332"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.560488 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "e7b0306d88784fe681642fa200f40f8d"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "537a009b7a9269c9bcee6dd90e78966c"
server_key_iv: "4404e934313df2a56629cadd03fd5332"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.561905 24434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:53.562884 24449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:53.563107 24434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:32:53.563256 24434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "e7b0306d88784fe681642fa200f40f8d"
format_stamp: "Formatted at 2026-08-09 00:32:53 on dist-test-slave-zpfg"
server_key: "537a009b7a9269c9bcee6dd90e78966c"
server_key_iv: "4404e934313df2a56629cadd03fd5332"
server_key_version: "encryptionkey@0"
I20260809 00:32:53.563354 24434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:53.581189 24434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:53.583503 24434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:53.583724 24434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:53.584026 24434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:53.584380 24434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:53.584416 24434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:53.584465 24434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:53.584496 24434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:53.591188 24434 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:33769
I20260809 00:32:53.591252 24562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:33769 every 8 connection(s)
I20260809 00:32:53.591598 24434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:32:53.593905 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 24434
I20260809 00:32:53.594013 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:32:53.594235 21671 external_mini_cluster.cc:1428] Setting key 79502ab150b843e396c447f32452bc46
Aug 09 00:32:53 dist-test-slave-zpfg krb5kdc[24058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235573, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:53.602098 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:53.592434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:38713 (local address 127.21.41.254:40193)
0809 00:32:53.592516 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:53.592518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:53.592738 (+ 220us) server_negotiation.cc:407] Connection header received
0809 00:32:53.593216 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:53.593218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:53.593229 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:53.593328 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:53.594700 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.595447 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.596439 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.596637 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.596693 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:53.599021 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:53.599023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:53.599023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:53.599030 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:53.600727 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:53.601145 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:53.601146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:53.601147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:53.601206 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:53.601464 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:53.601465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:53.601465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:53.601598 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:53.601681 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:53.601959 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 00:32:53.602031 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:32:53.602217 24565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:53.592427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:40193 (local address 127.21.41.195:38713)
0809 00:32:53.592608 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:53.592646 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:53.593131 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:53.593384 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:53.593387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:53.594139 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:53.594475 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:53.594480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.595527 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.595528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:53.596363 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:53.596365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:53.596713 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:53.596714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:53.596818 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:53.597293 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:53.597301 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:53.598927 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:53.600805 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:53.600811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:53.600813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:53.601039 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:53.601276 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:53.601277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:53.601277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:53.601378 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:53.601677 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:53.601682 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:53.601813 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 00:32:53.601899 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:32:53.602022 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260809 00:32:53.602748 24563 heartbeater.cc:347] Connected to a master server at 127.21.41.254:40193
I20260809 00:32:53.602857 24563 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:53.603037 24563 heartbeater.cc:511] Master 127.21.41.254:40193 requested a full tablet report, sending...
I20260809 00:32:53.603420 24103 ts_manager.cc:194] Registered new tserver with Master: e7b0306d88784fe681642fa200f40f8d (127.21.41.195:33769)
I20260809 00:32:53.603798 24103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:38713
I20260809 00:32:53.606158 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 00:32:53.619247 24143 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:32:53.618141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58202 (local address 127.21.41.254:40193)
0809 00:32:53.618233 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:53.618235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:53.618389 (+ 154us) server_negotiation.cc:407] Connection header received
0809 00:32:53.618457 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:53.618459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:53.618476 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:53.618543 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:53.619156 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58202: BlockingRecv error: recv got EOF from 127.0.0.1:58202 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 00:32:53.620075 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 24166
I20260809 00:32:53.625578 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 24300
I20260809 00:32:53.630331 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 24434
I20260809 00:32:53.634773 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 24071
2026-08-09T00:32:53Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1852 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:32:53 dist-test-slave-zpfg krb5kdc[24575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:32:53 dist-test-slave-zpfg krb5kdc[24575](info): set up 2 sockets
Aug 09 00:32:53 dist-test-slave-zpfg krb5kdc[24575](info): commencing operation
krb5kdc: starting...
W20260809 00:32:54.707780 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:32:54 dist-test-slave-zpfg krb5kdc[24575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:32:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:32:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:32:54.832988 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:39387
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:39387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:54.915467 24588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:54.915695 24588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:54.915719 24588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:54.916822 24588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:32:54.916859 24588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:54.916872 24588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:54.916885 24588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:32:54.916898 24588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:32:54.918334 24588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:39387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:39387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:54.918599 24588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:54.918764 24588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:54.923550 24596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:54.923574 24594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:54.923725 24588 server_base.cc:1034] running on GCE node
W20260809 00:32:54.923560 24593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:54.924057 24588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:54.924330 24588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:54.925486 24588 hybrid_clock.cc:648] HybridClock initialized: now 1786235574925462 us; error 38 us; skew 500 ppm
Aug 09 00:32:54 dist-test-slave-zpfg krb5kdc[24575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235574, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:54.928063 24588 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:32:54.928551 24588 webserver.cc:460] Webserver started at http://127.21.41.254:35171/ using document root <none> and password file <none>
I20260809 00:32:54.928788 24588 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:54.928854 24588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:54.928998 24588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:54.930081 24588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "e8133cc5a87f45be9208c92f79b0b4bb"
format_stamp: "Formatted at 2026-08-09 00:32:54 on dist-test-slave-zpfg"
server_key: "da6fb2c4713ce133780e93d57b9f926b"
server_key_iv: "c7f72271e65b0f39ea39360b22562959"
server_key_version: "encryptionkey@0"
I20260809 00:32:54.930495 24588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "e8133cc5a87f45be9208c92f79b0b4bb"
format_stamp: "Formatted at 2026-08-09 00:32:54 on dist-test-slave-zpfg"
server_key: "da6fb2c4713ce133780e93d57b9f926b"
server_key_iv: "c7f72271e65b0f39ea39360b22562959"
server_key_version: "encryptionkey@0"
I20260809 00:32:54.931846 24588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:54.932586 24603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:54.932785 24588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:54.932880 24588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "e8133cc5a87f45be9208c92f79b0b4bb"
format_stamp: "Formatted at 2026-08-09 00:32:54 on dist-test-slave-zpfg"
server_key: "da6fb2c4713ce133780e93d57b9f926b"
server_key_iv: "c7f72271e65b0f39ea39360b22562959"
server_key_version: "encryptionkey@0"
I20260809 00:32:54.932957 24588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:54.940835 24588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:54.943097 24588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:54.943261 24588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:54.947280 24588 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:39387
I20260809 00:32:54.947310 24655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:39387 every 8 connection(s)
I20260809 00:32:54.947842 24588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:32:54.948061 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 24588
I20260809 00:32:54.948167 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:32:54.948357 21671 external_mini_cluster.cc:1428] Setting key f04598ee5b16cb195224b9ff51b5b841
I20260809 00:32:54.948582 24656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:54.951427 24656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb: Bootstrap starting.
I20260809 00:32:54.952150 24656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:54.952437 24656 log.cc:824] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:54.953224 24656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb: No bootstrap required, opened a new log
Aug 09 00:32:54 dist-test-slave-zpfg krb5kdc[24575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:54.956000 24656 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8133cc5a87f45be9208c92f79b0b4bb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39387 } }
I20260809 00:32:54.956182 24656 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:54.956223 24656 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8133cc5a87f45be9208c92f79b0b4bb, State: Initialized, Role: FOLLOWER
I20260809 00:32:54.956348 24656 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8133cc5a87f45be9208c92f79b0b4bb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39387 } }
I20260809 00:32:54.956435 24656 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:32:54.956485 24656 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:32:54.956532 24656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:54.957203 24656 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8133cc5a87f45be9208c92f79b0b4bb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39387 } }
I20260809 00:32:54.957338 24656 leader_election.cc:302] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8133cc5a87f45be9208c92f79b0b4bb; no voters:
I20260809 00:32:54.957535 24656 leader_election.cc:288] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:32:54.957573 24661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:54.957711 24661 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [term 1 LEADER]: Becoming Leader. State: Replica: e8133cc5a87f45be9208c92f79b0b4bb, State: Running, Role: LEADER
I20260809 00:32:54.957851 24656 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:32:54.957875 24661 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8133cc5a87f45be9208c92f79b0b4bb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39387 } }
I20260809 00:32:54.958280 24661 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8133cc5a87f45be9208c92f79b0b4bb. Latest consensus state: current_term: 1 leader_uuid: "e8133cc5a87f45be9208c92f79b0b4bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8133cc5a87f45be9208c92f79b0b4bb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39387 } } }
I20260809 00:32:54.958498 24661 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:54.958294 24662 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8133cc5a87f45be9208c92f79b0b4bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8133cc5a87f45be9208c92f79b0b4bb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39387 } } }
I20260809 00:32:54.958763 24662 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:54.958771 24670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:32:54.959381 24670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:32:54.959888 24659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:54.948886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34940 (local address 127.21.41.254:39387)
0809 00:32:54.949082 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:54.949095 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:54.949120 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:32:54.949620 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:54.949627 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:54.949675 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:54.949863 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:54.950596 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:54.951457 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:54.952572 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:54.952992 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:54.953084 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:54.955827 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:54.955831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:54.955831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:54.955845 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:54.957820 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:54.958204 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:54.958207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:54.958207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:54.958326 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:54.958616 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:54.958617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:54.958617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:54.958941 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:54.959470 (+ 529us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:54.959545 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 00:32:54.959676 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260809 00:32:54.960806 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:32:54.962832 24670 catalog_manager.cc:1347] Generated new cluster ID: 4cf7f924e0e845108e63b30b36fa99ca
I20260809 00:32:54.962894 24670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:32:54.977419 24659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:54.973196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.21.41.254:39387)
0809 00:32:54.973246 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:54.973248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:54.973346 (+ 98us) server_negotiation.cc:407] Connection header received
0809 00:32:54.973396 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:54.973397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:54.973409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:54.973483 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:54.973865 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:54.974692 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:54.975716 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:54.976071 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:54.976147 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:54.976869 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:54.976870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:54.976871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:32:54.977141 (+ 270us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:54.977161 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:32:54.977168 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:54.977242 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:54.977301 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 00:32:54.977328 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:32:54.990950 24670 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:32:54.991721 24670 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:32:54.995100 24670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8133cc5a87f45be9208c92f79b0b4bb: Generated new TSK 0
I20260809 00:32:54.995307 24670 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:32:54.999315 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 24588
2026-08-09T00:32:55Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1362 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:32:55 dist-test-slave-zpfg krb5kdc[24689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:32:55 dist-test-slave-zpfg krb5kdc[24689](info): set up 2 sockets
Aug 09 00:32:55 dist-test-slave-zpfg krb5kdc[24689](info): commencing operation
krb5kdc: starting...
W20260809 00:32:56.043886 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:32:56 dist-test-slave-zpfg krb5kdc[24689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:32:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:32:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:32:56.169102 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:32773
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:32773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:56.251231 24702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:56.251436 24702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:56.251458 24702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:56.252534 24702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:32:56.252575 24702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:56.252590 24702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:56.252602 24702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:32:56.252614 24702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:32:56.253985 24702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:32773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:32773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:56.254172 24702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:56.254354 24702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:56.259199 24710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:56.259352 24702 server_base.cc:1034] running on GCE node
W20260809 00:32:56.259238 24707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:56.259202 24708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:56.259727 24702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:56.260021 24702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:56.261191 24702 hybrid_clock.cc:648] HybridClock initialized: now 1786235576261189 us; error 50 us; skew 500 ppm
Aug 09 00:32:56 dist-test-slave-zpfg krb5kdc[24689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235576, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:56.263813 24702 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:32:56.264286 24702 webserver.cc:460] Webserver started at http://127.21.41.254:45857/ using document root <none> and password file <none>
I20260809 00:32:56.264484 24702 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:56.264529 24702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:56.264631 24702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:56.265774 24702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "5bf2c977cb4048eea8886446f7a42ee5"
format_stamp: "Formatted at 2026-08-09 00:32:56 on dist-test-slave-zpfg"
server_key: "59788216dd2589f4260c32ba2d16f3c9"
server_key_iv: "af221ef3ff06b3fb9a164ba1127ab74f"
server_key_version: "encryptionkey@0"
I20260809 00:32:56.266129 24702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "5bf2c977cb4048eea8886446f7a42ee5"
format_stamp: "Formatted at 2026-08-09 00:32:56 on dist-test-slave-zpfg"
server_key: "59788216dd2589f4260c32ba2d16f3c9"
server_key_iv: "af221ef3ff06b3fb9a164ba1127ab74f"
server_key_version: "encryptionkey@0"
I20260809 00:32:56.267603 24702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:32:56.268400 24717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:56.268620 24702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:32:56.268757 24702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "5bf2c977cb4048eea8886446f7a42ee5"
format_stamp: "Formatted at 2026-08-09 00:32:56 on dist-test-slave-zpfg"
server_key: "59788216dd2589f4260c32ba2d16f3c9"
server_key_iv: "af221ef3ff06b3fb9a164ba1127ab74f"
server_key_version: "encryptionkey@0"
I20260809 00:32:56.268849 24702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:56.282454 24702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:56.284653 24702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:56.284839 24702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:56.288838 24702 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:32773
I20260809 00:32:56.288906 24769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:32773 every 8 connection(s)
I20260809 00:32:56.289273 24702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:32:56.290304 24770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:56.292589 24770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5: Bootstrap starting.
I20260809 00:32:56.293172 24770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:56.293401 24770 log.cc:824] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:56.294046 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 24702
I20260809 00:32:56.294162 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:32:56.294176 24770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5: No bootstrap required, opened a new log
I20260809 00:32:56.294432 21671 external_mini_cluster.cc:1428] Setting key 7352a83cf70fa3de0c261890073cd9e3
I20260809 00:32:56.296536 24770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf2c977cb4048eea8886446f7a42ee5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 32773 } }
I20260809 00:32:56.296727 24770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:56.296772 24770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf2c977cb4048eea8886446f7a42ee5, State: Initialized, Role: FOLLOWER
I20260809 00:32:56.296947 24770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf2c977cb4048eea8886446f7a42ee5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 32773 } }
I20260809 00:32:56.297046 24770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:32:56.297078 24770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:32:56.297127 24770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:56.297835 24770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf2c977cb4048eea8886446f7a42ee5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 32773 } }
I20260809 00:32:56.297943 24770 leader_election.cc:302] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bf2c977cb4048eea8886446f7a42ee5; no voters:
I20260809 00:32:56.298097 24770 leader_election.cc:288] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:32:56.298173 24775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:56.298370 24770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:32:56.298384 24775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [term 1 LEADER]: Becoming Leader. State: Replica: 5bf2c977cb4048eea8886446f7a42ee5, State: Running, Role: LEADER
I20260809 00:32:56.298497 24775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf2c977cb4048eea8886446f7a42ee5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 32773 } }
I20260809 00:32:56.298949 24776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bf2c977cb4048eea8886446f7a42ee5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf2c977cb4048eea8886446f7a42ee5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 32773 } } }
I20260809 00:32:56.299057 24776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:56.299252 24776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bf2c977cb4048eea8886446f7a42ee5. Latest consensus state: current_term: 1 leader_uuid: "5bf2c977cb4048eea8886446f7a42ee5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf2c977cb4048eea8886446f7a42ee5" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 32773 } } }
I20260809 00:32:56.299324 24784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:32:56.299378 24776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:56.299901 24784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:32:56.301546 24784 catalog_manager.cc:1347] Generated new cluster ID: 0d35303beeb94a9fa47724ce7ac3d901
I20260809 00:32:56.301602 24784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:32:56 dist-test-slave-zpfg krb5kdc[24689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:56.306990 24774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:56.294986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.21.41.254:32773)
0809 00:32:56.295190 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:56.295200 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:56.295233 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:32:56.295712 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:56.295720 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:56.295783 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:56.295929 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:56.296558 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:56.298422 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:56.299618 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:56.300230 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:56.300387 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:56.303278 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:56.303282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:56.303282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:56.303294 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:56.305334 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:56.305744 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:56.305747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:56.305747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:56.305854 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:56.306062 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:56.306063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:56.306064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:56.306408 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:56.306571 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:56.306645 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 00:32:56.306783 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":103,"threads_started":1}
I20260809 00:32:56.309960 24784 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:32:56.310555 24784 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:32:56.325335 24784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bf2c977cb4048eea8886446f7a42ee5: Generated new TSK 0
I20260809 00:32:56.325577 24784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:32:56.355170 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:32773
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:56.449522 24796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:56.449810 24796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:56.449855 24796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:56.451256 24796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:56.451329 24796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:56.451377 24796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:32:56.452984 24796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:32773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:56.453291 24796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:56.453554 24796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:56.459100 24801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:56.459110 24804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:56.459120 24802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:56.459383 24796 server_base.cc:1034] running on GCE node
I20260809 00:32:56.459712 24796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:56.459935 24796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:56.461087 24796 hybrid_clock.cc:648] HybridClock initialized: now 1786235576461075 us; error 28 us; skew 500 ppm
Aug 09 00:32:56 dist-test-slave-zpfg krb5kdc[24689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235576, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:56.464155 24796 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:32:56.464684 24796 webserver.cc:460] Webserver started at http://127.21.41.193:36327/ using document root <none> and password file <none>
I20260809 00:32:56.464931 24796 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:56.464979 24796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:56.465185 24796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:56.466426 24796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "b890a97e5e93448da1c036bbd4fad930"
format_stamp: "Formatted at 2026-08-09 00:32:56 on dist-test-slave-zpfg"
server_key: "894721e72dd6cfc62ea4f242f54f9196"
server_key_iv: "09f1a269e59f39f58c9ed669545605d0"
server_key_version: "encryptionkey@0"
I20260809 00:32:56.466851 24796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "b890a97e5e93448da1c036bbd4fad930"
format_stamp: "Formatted at 2026-08-09 00:32:56 on dist-test-slave-zpfg"
server_key: "894721e72dd6cfc62ea4f242f54f9196"
server_key_iv: "09f1a269e59f39f58c9ed669545605d0"
server_key_version: "encryptionkey@0"
I20260809 00:32:56.468310 24796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:32:56.469215 24811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:56.469436 24796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:56.469532 24796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "b890a97e5e93448da1c036bbd4fad930"
format_stamp: "Formatted at 2026-08-09 00:32:56 on dist-test-slave-zpfg"
server_key: "894721e72dd6cfc62ea4f242f54f9196"
server_key_iv: "09f1a269e59f39f58c9ed669545605d0"
server_key_version: "encryptionkey@0"
I20260809 00:32:56.469674 24796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:56.480921 24796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:56.483387 24796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:56.483512 24796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:56.483788 24796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:56.484169 24796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:56.484217 24796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:56.484241 24796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:56.484253 24796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:56.490583 24796 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:39433
I20260809 00:32:56.490624 24925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:39433 every 8 connection(s)
I20260809 00:32:56.491024 24796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:32:56.491423 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 24796
I20260809 00:32:56.491526 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:32:56.491752 21671 external_mini_cluster.cc:1428] Setting key a36d0bcd07fce5ec048ed868df65bbbc
Aug 09 00:32:56 dist-test-slave-zpfg krb5kdc[24689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235576, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
I20260809 00:32:56.501973 24774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:56.492059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:36409 (local address 127.21.41.254:32773)
0809 00:32:56.492188 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:56.492190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:56.492250 (+ 60us) server_negotiation.cc:407] Connection header received
0809 00:32:56.492930 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:56.492931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:56.492943 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:56.493004 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:56.494506 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:56.495035 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:56.496073 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:56.496418 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:56.496514 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:56.499017 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:56.499019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:56.499019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:56.499028 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:56.500546 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:56.501018 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:56.501020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:56.501021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:56.501093 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:56.501351 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:56.501352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:56.501352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:56.501468 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:56.501527 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:56.501814 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 00:32:56.501892 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260809 00:32:56.502144 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:56.491923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:32773 (local address 127.21.41.193:36409)
0809 00:32:56.492101 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:56.492154 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:56.492795 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:56.493083 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:56.493088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:56.493986 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:56.494301 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:56.494308 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:56.495133 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:56.495134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:56.495977 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:56.495979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:56.496517 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:56.496518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:56.496623 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:56.497243 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:56.497251 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:56.498878 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:56.500654 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:56.500662 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:56.500665 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:56.500906 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:56.501172 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:56.501173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:56.501173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:56.501279 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:56.501530 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:56.501535 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:56.501657 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 00:32:56.501747 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 00:32:56.501913 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260809 00:32:56.502723 24926 heartbeater.cc:347] Connected to a master server at 127.21.41.254:32773
I20260809 00:32:56.502825 24926 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:56.502978 24926 heartbeater.cc:511] Master 127.21.41.254:32773 requested a full tablet report, sending...
I20260809 00:32:56.503480 24734 ts_manager.cc:194] Registered new tserver with Master: b890a97e5e93448da1c036bbd4fad930 (127.21.41.193:39433)
I20260809 00:32:56.504132 24734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:36409
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
I20260809 00:32:57.505203 24926 heartbeater.cc:503] Master 127.21.41.254:32773 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
W20260809 00:32:57.766448 21671 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.41.194: real 1.274s user 0.000s sys 0.004s
I20260809 00:32:57.766852 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:32773
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:57.849538 24931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:57.849781 24931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:57.849815 24931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:57.851055 24931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:57.851105 24931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:57.851133 24931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:32:57.852670 24931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:32773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:57.852937 24931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:57.853175 24931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:57.858426 24939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:57.858426 24936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:57.858677 24931 server_base.cc:1034] running on GCE node
W20260809 00:32:57.858517 24937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:57.858942 24931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:57.859200 24931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:57.862411 24931 hybrid_clock.cc:648] HybridClock initialized: now 1786235577862380 us; error 64 us; skew 500 ppm
Aug 09 00:32:57 dist-test-slave-zpfg krb5kdc[24689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235577, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:57.869679 24931 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:32:57.870185 24931 webserver.cc:460] Webserver started at http://127.21.41.194:42389/ using document root <none> and password file <none>
I20260809 00:32:57.870497 24931 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:57.870569 24931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:57.870730 24931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:57.871874 24931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "74d9aa2e9fb849728a6ab4abfe782ddb"
format_stamp: "Formatted at 2026-08-09 00:32:57 on dist-test-slave-zpfg"
server_key: "46605fe30d5d28bca53da1761836b18a"
server_key_iv: "cf75d208db54af9cf4c450503e64bcc8"
server_key_version: "encryptionkey@0"
I20260809 00:32:57.872335 24931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "74d9aa2e9fb849728a6ab4abfe782ddb"
format_stamp: "Formatted at 2026-08-09 00:32:57 on dist-test-slave-zpfg"
server_key: "46605fe30d5d28bca53da1761836b18a"
server_key_iv: "cf75d208db54af9cf4c450503e64bcc8"
server_key_version: "encryptionkey@0"
I20260809 00:32:57.873829 24931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:57.874614 24946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:57.874790 24931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:32:57.874897 24931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "74d9aa2e9fb849728a6ab4abfe782ddb"
format_stamp: "Formatted at 2026-08-09 00:32:57 on dist-test-slave-zpfg"
server_key: "46605fe30d5d28bca53da1761836b18a"
server_key_iv: "cf75d208db54af9cf4c450503e64bcc8"
server_key_version: "encryptionkey@0"
I20260809 00:32:57.874969 24931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:57.896557 24931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:57.898710 24931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:57.898908 24931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:57.899152 24931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:57.899504 24931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:57.899570 24931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:57.899616 24931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:57.899643 24931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:57.905968 24931 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:38115
I20260809 00:32:57.906009 25059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:38115 every 8 connection(s)
I20260809 00:32:57.906392 24931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:32:57 dist-test-slave-zpfg krb5kdc[24689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235577, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:57.912396 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 24931
I20260809 00:32:57.912532 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:32:57.912738 21671 external_mini_cluster.cc:1428] Setting key 6c4a75c9277702968f178b5c321c9ba0
I20260809 00:32:57.916353 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:57.907188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:45007 (local address 127.21.41.254:32773)
0809 00:32:57.907367 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:57.907370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:57.907504 (+ 134us) server_negotiation.cc:407] Connection header received
0809 00:32:57.907999 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:57.908000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:57.908010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:57.908072 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:57.908992 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:57.909520 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:57.910556 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:57.910750 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:57.910804 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:57.912982 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:57.912984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:57.912985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:57.912992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:57.914830 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:57.915294 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:57.915295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:57.915296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:57.915362 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:57.915640 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:57.915641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:57.915641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:57.915787 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:57.915870 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:57.916157 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 00:32:57.916250 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":64,"threads_started":1}
I20260809 00:32:57.916560 25063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:57.907189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:32773 (local address 127.21.41.194:45007)
0809 00:32:57.907378 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:57.907416 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:57.907908 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:57.908126 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:57.908129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:57.908700 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:57.908907 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:57.908911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:57.909598 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:57.909599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:57.910471 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:57.910474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:57.910822 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:57.910823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:57.910917 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:57.911366 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:57.911374 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:57.912851 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:57.914917 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:57.914923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:57.914925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:57.915152 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:57.915436 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:57.915437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:57.915438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:57.915560 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:57.915859 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:57.915865 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:57.916007 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 00:32:57.916132 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 00:32:57.916309 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
I20260809 00:32:57.917058 25060 heartbeater.cc:347] Connected to a master server at 127.21.41.254:32773
I20260809 00:32:57.917168 25060 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:57.917352 25060 heartbeater.cc:511] Master 127.21.41.254:32773 requested a full tablet report, sending...
I20260809 00:32:57.917768 24734 ts_manager.cc:194] Registered new tserver with Master: 74d9aa2e9fb849728a6ab4abfe782ddb (127.21.41.194:38115)
I20260809 00:32:57.918380 24734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:45007
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
W20260809 00:32:58.221415 21671 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.41.195: real 0.308s user 0.000s sys 0.004s
I20260809 00:32:58.221815 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:32773
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:58.307912 25066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:58.308121 25066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:58.308144 25066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:58.309221 25066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:58.309257 25066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:58.309283 25066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:32:58.310828 25066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:32773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:58.311017 25066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:58.311190 25066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:58.316152 25074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:58.316345 25066 server_base.cc:1034] running on GCE node
W20260809 00:32:58.316180 25072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:58.316171 25071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:58.316675 25066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:58.316956 25066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:58.318130 25066 hybrid_clock.cc:648] HybridClock initialized: now 1786235578318123 us; error 35 us; skew 500 ppm
Aug 09 00:32:58 dist-test-slave-zpfg krb5kdc[24689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235578, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:58.320888 25066 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:32:58.321430 25066 webserver.cc:460] Webserver started at http://127.21.41.195:32929/ using document root <none> and password file <none>
I20260809 00:32:58.321668 25066 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:58.321745 25066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:58.321882 25066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:58.323060 25066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "63adf534bb6d4e97bc661fe60a53adb3"
format_stamp: "Formatted at 2026-08-09 00:32:58 on dist-test-slave-zpfg"
server_key: "47d53a5f893b174dddd28590c0f03e77"
server_key_iv: "f36f27fc55f1c2f5681fec99df1a5cc3"
server_key_version: "encryptionkey@0"
I20260809 00:32:58.323431 25066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "63adf534bb6d4e97bc661fe60a53adb3"
format_stamp: "Formatted at 2026-08-09 00:32:58 on dist-test-slave-zpfg"
server_key: "47d53a5f893b174dddd28590c0f03e77"
server_key_iv: "f36f27fc55f1c2f5681fec99df1a5cc3"
server_key_version: "encryptionkey@0"
I20260809 00:32:58.324746 25066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:32:58.325511 25081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:58.325678 25066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:58.325785 25066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "63adf534bb6d4e97bc661fe60a53adb3"
format_stamp: "Formatted at 2026-08-09 00:32:58 on dist-test-slave-zpfg"
server_key: "47d53a5f893b174dddd28590c0f03e77"
server_key_iv: "f36f27fc55f1c2f5681fec99df1a5cc3"
server_key_version: "encryptionkey@0"
I20260809 00:32:58.325861 25066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:58.347234 25066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:58.349795 25066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:58.349984 25066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:58.350255 25066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:58.350615 25066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:58.350673 25066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:58.350710 25066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:58.350740 25066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:58.356988 25066 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:41775
I20260809 00:32:58.357033 25194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:41775 every 8 connection(s)
I20260809 00:32:58.357447 25066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:32:58 dist-test-slave-zpfg krb5kdc[24689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235578, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:58.366803 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:58.358221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:42941 (local address 127.21.41.254:32773)
0809 00:32:58.358293 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:58.358295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:58.358541 (+ 246us) server_negotiation.cc:407] Connection header received
0809 00:32:58.359098 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:58.359100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:58.359118 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:58.359170 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:58.360136 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:58.360653 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:58.361683 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:58.361908 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:58.361983 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:58.364106 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:58.364108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:58.364108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:58.364116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:58.365322 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:58.365736 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:58.365738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:58.365738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:58.365806 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:58.366050 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:58.366051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:58.366052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:58.366172 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:58.366250 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:58.366572 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 00:32:58.366678 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:32:58.366918 25197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:58.358185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:32773 (local address 127.21.41.195:42941)
0809 00:32:58.358411 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:58.358447 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:58.358966 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:58.359248 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:58.359251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:58.359836 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:58.360027 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:58.360032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:58.360759 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:58.360760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:58.361587 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:58.361589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:58.362001 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:58.362002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:58.362123 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:58.362639 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:58.362648 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:58.364010 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:58.365421 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:58.365427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:58.365429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:58.365647 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:58.365876 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:58.365877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:58.365877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:58.365981 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:58.366240 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:58.366246 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:58.366441 (+ 195us) client_negotiation.cc:769] Sending connection context
0809 00:32:58.366551 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 00:32:58.366702 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":100,"threads_started":1}
I20260809 00:32:58.367389 25195 heartbeater.cc:347] Connected to a master server at 127.21.41.254:32773
I20260809 00:32:58.367511 25195 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:58.367545 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 25066
I20260809 00:32:58.367633 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:32:58.367729 25195 heartbeater.cc:511] Master 127.21.41.254:32773 requested a full tablet report, sending...
I20260809 00:32:58.367851 21671 external_mini_cluster.cc:1428] Setting key 6dff1075a3113d67f7f8afbaeada145d
I20260809 00:32:58.368233 24734 ts_manager.cc:194] Registered new tserver with Master: 63adf534bb6d4e97bc661fe60a53adb3 (127.21.41.195:41775)
I20260809 00:32:58.368724 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:32:58.368825 24734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:42941
Aug 09 00:32:58 dist-test-slave-zpfg krb5kdc[24689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235578, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 00:32:58 dist-test-slave-zpfg krb5kdc[24689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235578, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:58.392673 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:58.385203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42898 (local address 127.21.41.254:32773)
0809 00:32:58.385273 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:58.385276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:58.385401 (+ 125us) server_negotiation.cc:407] Connection header received
0809 00:32:58.385501 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:58.385502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:58.385519 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:58.385688 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:58.386299 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:58.386895 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:58.387679 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:58.387878 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:58.387958 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:58.390150 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:58.390152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:58.390152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:58.390159 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:58.391467 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:58.391839 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:58.391840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:58.391841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:58.391900 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:58.392099 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:58.392100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:58.392100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:58.392265 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:58.392383 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:58.392486 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 00:32:58.392564 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 00:32:58.392921 24734 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42898
I20260809 00:32:58.393384 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 24796
I20260809 00:32:58.398236 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 24931
I20260809 00:32:58.402740 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 25066
I20260809 00:32:58.407084 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 24702
2026-08-09T00:32:58Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3408 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:32:58 dist-test-slave-zpfg krb5kdc[25207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:32:58 dist-test-slave-zpfg krb5kdc[25207](info): set up 2 sockets
Aug 09 00:32:58 dist-test-slave-zpfg krb5kdc[25207](info): commencing operation
krb5kdc: starting...
W20260809 00:32:59.447917 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:32:59 dist-test-slave-zpfg krb5kdc[25207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:32:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:32:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:32:59.571151 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:34879
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:42985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:34879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:59.654024 25220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:59.654238 25220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:59.654260 25220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:59.655467 25220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:32:59.655541 25220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:59.655596 25220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:59.655618 25220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:32:59.655651 25220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:32:59.657186 25220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:34879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:34879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:59.657387 25220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:59.657567 25220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:59.662492 25228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:59.662652 25220 server_base.cc:1034] running on GCE node
W20260809 00:32:59.662532 25226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:59.662532 25225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:59.662941 25220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:59.663241 25220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:59.664386 25220 hybrid_clock.cc:648] HybridClock initialized: now 1786235579664366 us; error 39 us; skew 500 ppm
Aug 09 00:32:59 dist-test-slave-zpfg krb5kdc[25207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235579, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:59.666883 25220 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:32:59.667369 25220 webserver.cc:460] Webserver started at http://127.21.41.254:45849/ using document root <none> and password file <none>
I20260809 00:32:59.667599 25220 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:59.667644 25220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:59.667811 25220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:59.668883 25220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "caf86fc666d14111a37884675b48bee7"
format_stamp: "Formatted at 2026-08-09 00:32:59 on dist-test-slave-zpfg"
server_key: "2d60a720aa19c8335a64bb1b4e170a7d"
server_key_iv: "9a7908209e63ee1bc0638dec0505194d"
server_key_version: "encryptionkey@0"
I20260809 00:32:59.669251 25220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "caf86fc666d14111a37884675b48bee7"
format_stamp: "Formatted at 2026-08-09 00:32:59 on dist-test-slave-zpfg"
server_key: "2d60a720aa19c8335a64bb1b4e170a7d"
server_key_iv: "9a7908209e63ee1bc0638dec0505194d"
server_key_version: "encryptionkey@0"
I20260809 00:32:59.670598 25220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:32:59.671334 25235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:59.671512 25220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:59.671602 25220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "caf86fc666d14111a37884675b48bee7"
format_stamp: "Formatted at 2026-08-09 00:32:59 on dist-test-slave-zpfg"
server_key: "2d60a720aa19c8335a64bb1b4e170a7d"
server_key_iv: "9a7908209e63ee1bc0638dec0505194d"
server_key_version: "encryptionkey@0"
I20260809 00:32:59.671674 25220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:59.679196 25220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:59.681069 25220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:59.681226 25220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:59.685521 25220 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:34879
I20260809 00:32:59.685552 25287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:34879 every 8 connection(s)
I20260809 00:32:59.686136 25220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:32:59.686865 25288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:32:59.689112 25288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7: Bootstrap starting.
I20260809 00:32:59.689713 25288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7: Neither blocks nor log segments found. Creating new log.
I20260809 00:32:59.690022 25288 log.cc:824] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:32:59.690752 25288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7: No bootstrap required, opened a new log
I20260809 00:32:59.692329 25288 raft_consensus.cc:348] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf86fc666d14111a37884675b48bee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 34879 } }
I20260809 00:32:59.692463 25288 raft_consensus.cc:374] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:32:59.692485 25288 raft_consensus.cc:729] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf86fc666d14111a37884675b48bee7, State: Initialized, Role: FOLLOWER
I20260809 00:32:59.692651 25288 consensus_queue.cc:260] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf86fc666d14111a37884675b48bee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 34879 } }
I20260809 00:32:59.692721 25288 raft_consensus.cc:388] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:32:59.692773 25288 raft_consensus.cc:482] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:32:59.692821 25288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:32:59.693713 25288 raft_consensus.cc:504] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf86fc666d14111a37884675b48bee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 34879 } }
I20260809 00:32:59.693825 25288 leader_election.cc:302] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caf86fc666d14111a37884675b48bee7; no voters:
I20260809 00:32:59.694000 25288 leader_election.cc:288] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:32:59.694041 25291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:32:59.694190 25291 raft_consensus.cc:686] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [term 1 LEADER]: Becoming Leader. State: Replica: caf86fc666d14111a37884675b48bee7, State: Running, Role: LEADER
I20260809 00:32:59.694270 25288 sys_catalog.cc:564] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:32:59.694324 25291 consensus_queue.cc:237] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf86fc666d14111a37884675b48bee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 34879 } }
I20260809 00:32:59.694723 25292 sys_catalog.cc:455] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caf86fc666d14111a37884675b48bee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf86fc666d14111a37884675b48bee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 34879 } } }
I20260809 00:32:59.694844 25292 sys_catalog.cc:458] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:59.694756 25293 sys_catalog.cc:455] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader caf86fc666d14111a37884675b48bee7. Latest consensus state: current_term: 1 leader_uuid: "caf86fc666d14111a37884675b48bee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf86fc666d14111a37884675b48bee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 34879 } } }
I20260809 00:32:59.695014 25293 sys_catalog.cc:458] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:32:59.695319 25299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:32:59.695823 25299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:32:59.696242 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 25220
I20260809 00:32:59.696354 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:32:59.696555 21671 external_mini_cluster.cc:1428] Setting key 074a8d0a8033e219704e9131643d2057
I20260809 00:32:59.697616 25299 catalog_manager.cc:1347] Generated new cluster ID: 26ee9eb9647d420bbcfa4a5894586769
I20260809 00:32:59.697670 25299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:32:59 dist-test-slave-zpfg krb5kdc[25207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:59.704200 25299 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:32:59.705057 25299 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:32:59.707830 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:59.697051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.21.41.254:34879)
0809 00:32:59.697325 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:59.697334 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:59.697368 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:32:59.697482 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:59.697500 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:59.697554 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:59.698563 (+ 1009us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:59.699206 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:59.700124 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:59.700875 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:59.701315 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:59.701433 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:59.703788 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:59.703792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:59.703792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:59.703804 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:59.706115 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:59.706516 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:59.706520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:59.706520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:59.706630 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:59.706838 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:59.706839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:59.706839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:59.707109 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:59.707290 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:59.707356 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 00:32:59.707602 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":87,"threads_started":1}
I20260809 00:32:59.710315 25299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caf86fc666d14111a37884675b48bee7: Generated new TSK 0
I20260809 00:32:59.710487 25299 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:32:59.739184 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:34879
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:42985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:59.821414 25314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:59.821619 25314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:59.821641 25314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:59.822824 25314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:59.822865 25314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:59.822892 25314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:32:59.824419 25314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:34879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:59.824601 25314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:59.824785 25314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:59.829808 25319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:59.829834 25320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:59.829878 25322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:59.830047 25314 server_base.cc:1034] running on GCE node
I20260809 00:32:59.830394 25314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:59.830662 25314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:59.831806 25314 hybrid_clock.cc:648] HybridClock initialized: now 1786235579831782 us; error 35 us; skew 500 ppm
Aug 09 00:32:59 dist-test-slave-zpfg krb5kdc[25207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235579, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:59.834494 25314 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:32:59.834890 25314 webserver.cc:460] Webserver started at http://127.21.41.193:32883/ using document root <none> and password file <none>
I20260809 00:32:59.835068 25314 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:59.835107 25314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:59.835196 25314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:59.836212 25314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "2034bd8bf7f14a71b4f316b79c7928b0"
format_stamp: "Formatted at 2026-08-09 00:32:59 on dist-test-slave-zpfg"
server_key: "0818cca2e6c0e5a1c5cde92d136c37b0"
server_key_iv: "f8325ca405c4285df2d6aa4f7c303aef"
server_key_version: "encryptionkey@0"
I20260809 00:32:59.836521 25314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "2034bd8bf7f14a71b4f316b79c7928b0"
format_stamp: "Formatted at 2026-08-09 00:32:59 on dist-test-slave-zpfg"
server_key: "0818cca2e6c0e5a1c5cde92d136c37b0"
server_key_iv: "f8325ca405c4285df2d6aa4f7c303aef"
server_key_version: "encryptionkey@0"
I20260809 00:32:59.837805 25314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:59.838518 25329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:59.838671 25314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:59.838783 25314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "2034bd8bf7f14a71b4f316b79c7928b0"
format_stamp: "Formatted at 2026-08-09 00:32:59 on dist-test-slave-zpfg"
server_key: "0818cca2e6c0e5a1c5cde92d136c37b0"
server_key_iv: "f8325ca405c4285df2d6aa4f7c303aef"
server_key_version: "encryptionkey@0"
I20260809 00:32:59.838841 25314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:32:59.848075 25314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:32:59.849853 25314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:32:59.849964 25314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:32:59.850149 25314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:32:59.850466 25314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:32:59.850502 25314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:59.850525 25314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:32:59.850538 25314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:59.856091 25314 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:34577
I20260809 00:32:59.856177 25442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:34577 every 8 connection(s)
I20260809 00:32:59.856503 25314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:32:59 dist-test-slave-zpfg krb5kdc[25207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235579, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:32:59.864238 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 25314
I20260809 00:32:59.864360 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:32:59.864562 21671 external_mini_cluster.cc:1428] Setting key 2232e688cceacf8befe7c30739461d9a
I20260809 00:32:59.866515 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:59.857277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:57665 (local address 127.21.41.254:34879)
0809 00:32:59.857352 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:32:59.857354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:32:59.857594 (+ 240us) server_negotiation.cc:407] Connection header received
0809 00:32:59.858603 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:32:59.858604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:32:59.858614 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:32:59.858667 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:32:59.859308 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:59.859825 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:59.860840 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:32:59.861051 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:32:59.861120 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:59.863289 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:32:59.863291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:32:59.863291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:32:59.863299 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:32:59.864784 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:59.865366 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:59.865368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:59.865368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:59.865450 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:32:59.865733 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:32:59.865734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:32:59.865734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:32:59.865894 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:32:59.865973 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:32:59.866217 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 00:32:59.866372 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:32:59.866606 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:32:59.857280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:34879 (local address 127.21.41.193:57665)
0809 00:32:59.857472 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:32:59.857509 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:32:59.858506 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:32:59.858754 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:32:59.858758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:32:59.859018 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:32:59.859224 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:59.859228 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:59.859904 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:59.859905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:59.860741 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:32:59.860743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:32:59.861140 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:32:59.861141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:32:59.861255 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:32:59.861739 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:32:59.861747 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:32:59.863200 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:32:59.864887 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:59.864895 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:59.864898 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:59.865227 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:59.865529 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:32:59.865530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:32:59.865530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:32:59.865661 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:32:59.865963 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:32:59.865968 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:32:59.866086 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 00:32:59.866168 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 00:32:59.866318 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260809 00:32:59.867656 25443 heartbeater.cc:347] Connected to a master server at 127.21.41.254:34879
I20260809 00:32:59.867781 25443 heartbeater.cc:464] Registering TS with master...
I20260809 00:32:59.868014 25443 heartbeater.cc:511] Master 127.21.41.254:34879 requested a full tablet report, sending...
I20260809 00:32:59.868575 25252 ts_manager.cc:194] Registered new tserver with Master: 2034bd8bf7f14a71b4f316b79c7928b0 (127.21.41.193:34577)
I20260809 00:32:59.869195 25252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:57665
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:32:59.892616 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:34879
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:42985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:32:59.974812 25448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:32:59.975030 25448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:32:59.975068 25448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:32:59.976256 25448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:32:59.976320 25448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:32:59.976366 25448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:32:59.977874 25448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:34879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:32:59.978132 25448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:32:59.978401 25448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:32:59.983398 25456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:59.983464 25454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:32:59.983398 25453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:32:59.983661 25448 server_base.cc:1034] running on GCE node
I20260809 00:32:59.983918 25448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:32:59.984165 25448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:32:59.985324 25448 hybrid_clock.cc:648] HybridClock initialized: now 1786235579985317 us; error 31 us; skew 500 ppm
Aug 09 00:32:59 dist-test-slave-zpfg krb5kdc[25207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235579, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:32:59.987982 25448 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:32:59.988370 25448 webserver.cc:460] Webserver started at http://127.21.41.194:44745/ using document root <none> and password file <none>
I20260809 00:32:59.988566 25448 fs_manager.cc:359] Metadata directory not provided
I20260809 00:32:59.988606 25448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:32:59.988693 25448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:32:59.989683 25448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "a2cf975f949949f8b8d266c0713328b3"
format_stamp: "Formatted at 2026-08-09 00:32:59 on dist-test-slave-zpfg"
server_key: "d1e72e5bde529e83e3e6f0b87e45b451"
server_key_iv: "d23859190a85217eb370ba3b0c9ce168"
server_key_version: "encryptionkey@0"
I20260809 00:32:59.989995 25448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "a2cf975f949949f8b8d266c0713328b3"
format_stamp: "Formatted at 2026-08-09 00:32:59 on dist-test-slave-zpfg"
server_key: "d1e72e5bde529e83e3e6f0b87e45b451"
server_key_iv: "d23859190a85217eb370ba3b0c9ce168"
server_key_version: "encryptionkey@0"
I20260809 00:32:59.991353 25448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:32:59.992033 25463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:32:59.992189 25448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:32:59.992277 25448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "a2cf975f949949f8b8d266c0713328b3"
format_stamp: "Formatted at 2026-08-09 00:32:59 on dist-test-slave-zpfg"
server_key: "d1e72e5bde529e83e3e6f0b87e45b451"
server_key_iv: "d23859190a85217eb370ba3b0c9ce168"
server_key_version: "encryptionkey@0"
I20260809 00:32:59.992329 25448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:00.004132 25448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:00.006242 25448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:00.006426 25448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:00.006632 25448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:00.006948 25448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:00.006982 25448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:00.007028 25448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:00.007056 25448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:00.012792 25448 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:38703
I20260809 00:33:00.012877 25576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:38703 every 8 connection(s)
I20260809 00:33:00.013213 25448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:33:00.017747 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 25448
I20260809 00:33:00.017874 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:33:00.018057 21671 external_mini_cluster.cc:1428] Setting key fbcd0471f478b4a9c9ccda92546f9e7b
Aug 09 00:33:00 dist-test-slave-zpfg krb5kdc[25207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235579, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:00.023216 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:00.014009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:58839 (local address 127.21.41.254:34879)
0809 00:33:00.014056 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:00.014058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:00.014248 (+ 190us) server_negotiation.cc:407] Connection header received
0809 00:33:00.015185 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:00.015186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:00.015194 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:00.015260 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:00.015868 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.016363 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.017308 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.017490 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.017536 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:00.020270 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:00.020271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:00.020272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:00.020280 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:00.021727 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:00.022152 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:00.022153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:00.022154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:00.022221 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:00.022524 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:00.022525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:00.022525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:00.022643 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:00.022722 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:00.023020 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 00:33:00.023108 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:33:00.023414 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:00.013937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:34879 (local address 127.21.41.194:58839)
0809 00:33:00.014111 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:00.014154 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:00.015095 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:00.015322 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:00.015325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:00.015597 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:00.015793 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:00.015797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.016444 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.016445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:00.017224 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:00.017226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.017587 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.017588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:00.017738 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:00.018404 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:00.018416 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:00.020170 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:00.021831 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:00.021839 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:00.021841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:00.022071 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:00.022296 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:00.022297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:00.022298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:00.022454 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:00.022736 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:00.022742 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:00.022885 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 00:33:00.022988 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 00:33:00.023153 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260809 00:33:00.024037 25577 heartbeater.cc:347] Connected to a master server at 127.21.41.254:34879
I20260809 00:33:00.024122 25577 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:00.024338 25577 heartbeater.cc:511] Master 127.21.41.254:34879 requested a full tablet report, sending...
I20260809 00:33:00.024736 25252 ts_manager.cc:194] Registered new tserver with Master: a2cf975f949949f8b8d266c0713328b3 (127.21.41.194:38703)
I20260809 00:33:00.025264 25252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:58839
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:33:00.045230 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:34879
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:42985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:00.130038 25582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:00.130241 25582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:00.130271 25582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:00.131461 25582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:00.131505 25582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:00.131531 25582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:00.132992 25582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:34879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:00.133184 25582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:00.133388 25582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:00.138453 25588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:00.138604 25582 server_base.cc:1034] running on GCE node
W20260809 00:33:00.138490 25587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:00.138453 25590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:00.138886 25582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:00.139137 25582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:00.140311 25582 hybrid_clock.cc:648] HybridClock initialized: now 1786235580140298 us; error 43 us; skew 500 ppm
Aug 09 00:33:00 dist-test-slave-zpfg krb5kdc[25207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235580, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:00.143185 25582 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:33:00.143651 25582 webserver.cc:460] Webserver started at http://127.21.41.195:35137/ using document root <none> and password file <none>
I20260809 00:33:00.143887 25582 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:00.143956 25582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:00.144106 25582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:00.145149 25582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "45dd1b503ab24da3a25fa26a210d0e06"
format_stamp: "Formatted at 2026-08-09 00:33:00 on dist-test-slave-zpfg"
server_key: "555bd8ff03efb69bb0c9383e2c72638d"
server_key_iv: "8f9527b99d0b2809155dbb1537d567a2"
server_key_version: "encryptionkey@0"
I20260809 00:33:00.145521 25582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "45dd1b503ab24da3a25fa26a210d0e06"
format_stamp: "Formatted at 2026-08-09 00:33:00 on dist-test-slave-zpfg"
server_key: "555bd8ff03efb69bb0c9383e2c72638d"
server_key_iv: "8f9527b99d0b2809155dbb1537d567a2"
server_key_version: "encryptionkey@0"
I20260809 00:33:00.146937 25582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:33:00.147651 25597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:00.147809 25582 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:00.147907 25582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "45dd1b503ab24da3a25fa26a210d0e06"
format_stamp: "Formatted at 2026-08-09 00:33:00 on dist-test-slave-zpfg"
server_key: "555bd8ff03efb69bb0c9383e2c72638d"
server_key_iv: "8f9527b99d0b2809155dbb1537d567a2"
server_key_version: "encryptionkey@0"
I20260809 00:33:00.147982 25582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:00.174034 25582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:00.176539 25582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:00.176698 25582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:00.176970 25582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:00.177340 25582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:00.177378 25582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:00.177402 25582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:00.177443 25582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:00.184113 25582 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:39655
I20260809 00:33:00.184161 25710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:39655 every 8 connection(s)
I20260809 00:33:00.184496 25582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:33:00 dist-test-slave-zpfg krb5kdc[25207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235580, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:00.190778 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 25582
I20260809 00:33:00.190886 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:33:00.191087 21671 external_mini_cluster.cc:1428] Setting key 7f71f2d529c59cb19ae31214065849a7
I20260809 00:33:00.193729 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:00.185244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:44487 (local address 127.21.41.254:34879)
0809 00:33:00.185293 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:00.185294 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:00.185492 (+ 198us) server_negotiation.cc:407] Connection header received
0809 00:33:00.186456 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:00.186458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:00.186468 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:00.186517 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:00.187097 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.187588 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.188528 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.188716 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.188777 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:00.191010 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:00.191012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:00.191012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:00.191020 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:00.192598 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:00.192952 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:00.192953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:00.192953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:00.193003 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:00.193225 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:00.193225 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:00.193225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:00.193321 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:00.193369 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:00.193574 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 00:33:00.193630 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:33:00.194002 25713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:00.185237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:34879 (local address 127.21.41.195:44487)
0809 00:33:00.185375 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:00.185409 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:00.186318 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:00.186579 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:00.186582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:00.186834 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:00.187026 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:00.187030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.187662 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.187663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:00.188450 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:00.188451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.188795 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.188796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:00.188893 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:00.189417 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:00.189425 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:00.190909 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:00.192668 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:00.192673 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:00.192674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:00.192888 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:00.193059 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:00.193059 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:00.193060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:00.193158 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:00.193374 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:00.193378 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:00.193484 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 00:33:00.193562 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:33:00.193736 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260809 00:33:00.194516 25711 heartbeater.cc:347] Connected to a master server at 127.21.41.254:34879
I20260809 00:33:00.194612 25711 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:00.194854 25711 heartbeater.cc:511] Master 127.21.41.254:34879 requested a full tablet report, sending...
I20260809 00:33:00.195240 25252 ts_manager.cc:194] Registered new tserver with Master: 45dd1b503ab24da3a25fa26a210d0e06 (127.21.41.195:39655)
I20260809 00:33:00.195675 25252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:44487
I20260809 00:33:00.202939 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 00:33:00 dist-test-slave-zpfg krb5kdc[25207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:33:00 dist-test-slave-zpfg krb5kdc[25207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.193@KRBTEST.COM
I20260809 00:33:00.225786 25718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:00.218346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49450 (local address 127.21.41.193:34577)
0809 00:33:00.218535 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:00.218540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:00.218550 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:00.218584 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:00.218586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:00.218601 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:00.218667 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:00.219193 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.219945 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.220671 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.221037 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.221093 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:00.223352 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:00.223361 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:00.223361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:00.223367 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:00.224767 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:00.225071 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:00.225074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:00.225074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:00.225139 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:00.225302 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:00.225302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:00.225303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:00.225431 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:00.225545 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:00.225598 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 00:33:00.225660 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
Aug 09 00:33:00 dist-test-slave-zpfg krb5kdc[25207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:00.233398 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:00.227239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52454 (local address 127.21.41.254:34879)
0809 00:33:00.227283 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:00.227285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:00.227318 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:33:00.227339 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:00.227340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:00.227348 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:00.227386 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:00.227863 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.228336 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.228979 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:00.229152 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:00.229200 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:00.231242 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:00.231244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:00.231244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:00.231251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:00.232421 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:00.232724 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:00.232725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:00.232725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:00.232775 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:00.232931 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:00.232931 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:00.232931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:00.233072 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:00.233147 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:00.233232 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 00:33:00.233291 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 00:33:00.233686 25252 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52454
I20260809 00:33:00.234194 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 25314
I20260809 00:33:00.239111 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 25448
I20260809 00:33:00.243322 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 25582
I20260809 00:33:00.247496 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 25220
2026-08-09T00:33:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1840 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:33:00 dist-test-slave-zpfg krb5kdc[25723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:33:00 dist-test-slave-zpfg krb5kdc[25723](info): set up 2 sockets
Aug 09 00:33:00 dist-test-slave-zpfg krb5kdc[25723](info): commencing operation
krb5kdc: starting...
W20260809 00:33:01.287976 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:33:01 dist-test-slave-zpfg krb5kdc[25723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:33:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:33:01.410844 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:46507
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:46507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:01.492427 25736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:01.492645 25736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:01.492668 25736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:01.493883 25736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:01.493930 25736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:01.493944 25736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:01.493957 25736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:01.494011 25736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:01.495640 25736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:46507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:46507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:01.495947 25736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:01.496202 25736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:01.501072 25742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:01.501085 25743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:01.501092 25745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:01.501219 25736 server_base.cc:1034] running on GCE node
I20260809 00:33:01.501593 25736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:01.501902 25736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:01.503067 25736 hybrid_clock.cc:648] HybridClock initialized: now 1786235581503055 us; error 46 us; skew 500 ppm
Aug 09 00:33:01 dist-test-slave-zpfg krb5kdc[25723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:01.505746 25736 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:33:01.506107 25736 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:01.506155 25736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:01.506249 25736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:01.507383 25736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "040ba4e93e964bd486b2df864e3c785f"
format_stamp: "Formatted at 2026-08-09 00:33:01 on dist-test-slave-zpfg"
server_key: "7c55cd9295817a1c8881517bb6fc2fe1"
server_key_iv: "dba8dca13d3a479429d03aa13c86e6d5"
server_key_version: "encryptionkey@0"
I20260809 00:33:01.507687 25736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "040ba4e93e964bd486b2df864e3c785f"
format_stamp: "Formatted at 2026-08-09 00:33:01 on dist-test-slave-zpfg"
server_key: "7c55cd9295817a1c8881517bb6fc2fe1"
server_key_iv: "dba8dca13d3a479429d03aa13c86e6d5"
server_key_version: "encryptionkey@0"
I20260809 00:33:01.509184 25736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:33:01.510025 25751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:01.510231 25736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:01.510337 25736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "040ba4e93e964bd486b2df864e3c785f"
format_stamp: "Formatted at 2026-08-09 00:33:01 on dist-test-slave-zpfg"
server_key: "7c55cd9295817a1c8881517bb6fc2fe1"
server_key_iv: "dba8dca13d3a479429d03aa13c86e6d5"
server_key_version: "encryptionkey@0"
I20260809 00:33:01.510453 25736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:01.521528 25736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:01.523905 25736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:01.524070 25736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:01.527972 25736 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:46507
I20260809 00:33:01.527998 25803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:46507 every 8 connection(s)
I20260809 00:33:01.528294 25736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:01.529299 25804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:01.531672 25804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f: Bootstrap starting.
I20260809 00:33:01.532310 25804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:01.532614 25804 log.cc:824] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:01.533303 25804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f: No bootstrap required, opened a new log
I20260809 00:33:01.534960 25804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040ba4e93e964bd486b2df864e3c785f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46507 } }
I20260809 00:33:01.535092 25804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:01.535113 25804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 040ba4e93e964bd486b2df864e3c785f, State: Initialized, Role: FOLLOWER
I20260809 00:33:01.535277 25804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040ba4e93e964bd486b2df864e3c785f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46507 } }
I20260809 00:33:01.535347 25804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:01.535389 25804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:01.535439 25804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:01.535972 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 25736
I20260809 00:33:01.536100 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:01.536283 21671 external_mini_cluster.cc:1428] Setting key 567fe7b8bfab5036a2ab7b519cd605cb
I20260809 00:33:01.536331 25804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040ba4e93e964bd486b2df864e3c785f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46507 } }
I20260809 00:33:01.536481 25804 leader_election.cc:302] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 040ba4e93e964bd486b2df864e3c785f; no voters:
I20260809 00:33:01.536677 25804 leader_election.cc:288] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:01.536715 25807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:01.536831 25807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [term 1 LEADER]: Becoming Leader. State: Replica: 040ba4e93e964bd486b2df864e3c785f, State: Running, Role: LEADER
I20260809 00:33:01.536978 25807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040ba4e93e964bd486b2df864e3c785f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46507 } }
I20260809 00:33:01.537132 25804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:01.537312 25810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 040ba4e93e964bd486b2df864e3c785f. Latest consensus state: current_term: 1 leader_uuid: "040ba4e93e964bd486b2df864e3c785f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040ba4e93e964bd486b2df864e3c785f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46507 } } }
I20260809 00:33:01.537314 25809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "040ba4e93e964bd486b2df864e3c785f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040ba4e93e964bd486b2df864e3c785f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46507 } } }
I20260809 00:33:01.537448 25810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:01.537456 25809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:01.537734 25813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:01.538378 25813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:01.540021 25813 catalog_manager.cc:1347] Generated new cluster ID: a77db5f934be4ddbbf3a1a34edf8b646
I20260809 00:33:01.540071 25813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:33:01 dist-test-slave-zpfg krb5kdc[25723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:01.548691 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:01.536967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.21.41.254:46507)
0809 00:33:01.537169 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:01.537178 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:01.537211 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:33:01.537325 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:01.537331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:01.537398 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:01.538411 (+ 1013us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:01.538532 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:01.539808 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:01.540887 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:01.541604 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:01.542027 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:01.544556 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:01.544560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:01.544561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:01.544573 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:01.546642 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:01.547062 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:01.547065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:01.547065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:01.547182 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:01.547421 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:01.547422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:01.547422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:01.547731 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:01.548274 (+ 543us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:01.548347 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 00:33:01.548485 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":71,"threads_started":1}
I20260809 00:33:01.553076 25813 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:01.553656 25813 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:01.562961 25813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f: Generated new TSK 0
I20260809 00:33:01.563169 25813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:33:01.593909 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46507
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:01.676960 25831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:01.677175 25831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:01.677199 25831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:01.678390 25831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:01.678431 25831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:01.678486 25831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:33:01.680017 25831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:01.680287 25831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:01.680523 25831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:01.685480 25837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:01.685499 25836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:01.685499 25839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:01.685750 25831 server_base.cc:1034] running on GCE node
I20260809 00:33:01.685959 25831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:01.686201 25831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:01.687367 25831 hybrid_clock.cc:648] HybridClock initialized: now 1786235581687341 us; error 39 us; skew 500 ppm
Aug 09 00:33:01 dist-test-slave-zpfg krb5kdc[25723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:01.689913 25831 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:33:01.690210 25831 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:01.690255 25831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:01.690400 25831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:01.691468 25831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "643162250b9a4e4db02eb7793ff5a67c"
format_stamp: "Formatted at 2026-08-09 00:33:01 on dist-test-slave-zpfg"
server_key: "69f1398c1b8bd0708aee3ef2a5cd258c"
server_key_iv: "b535f85ddb21478df4d21c2c9caa270f"
server_key_version: "encryptionkey@0"
I20260809 00:33:01.691790 25831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "643162250b9a4e4db02eb7793ff5a67c"
format_stamp: "Formatted at 2026-08-09 00:33:01 on dist-test-slave-zpfg"
server_key: "69f1398c1b8bd0708aee3ef2a5cd258c"
server_key_iv: "b535f85ddb21478df4d21c2c9caa270f"
server_key_version: "encryptionkey@0"
I20260809 00:33:01.693013 25831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:33:01.693676 25845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:01.693871 25831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:01.693958 25831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "643162250b9a4e4db02eb7793ff5a67c"
format_stamp: "Formatted at 2026-08-09 00:33:01 on dist-test-slave-zpfg"
server_key: "69f1398c1b8bd0708aee3ef2a5cd258c"
server_key_iv: "b535f85ddb21478df4d21c2c9caa270f"
server_key_version: "encryptionkey@0"
I20260809 00:33:01.694015 25831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:01.709218 25831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:01.711071 25831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:01.711184 25831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:01.711371 25831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:01.711680 25831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:01.711712 25831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:01.711776 25831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:01.711807 25831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:01.717655 25831 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:39573
I20260809 00:33:01.717721 25959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:39573 every 8 connection(s)
I20260809 00:33:01.718004 25831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:33:01.719221 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 25831
I20260809 00:33:01.719333 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:33:01.719501 21671 external_mini_cluster.cc:1428] Setting key 43db13a631a1fa5aa0c414d88fe70fa6
Aug 09 00:33:01 dist-test-slave-zpfg krb5kdc[25723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
I20260809 00:33:01.728593 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:01.718775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:50907 (local address 127.21.41.254:46507)
0809 00:33:01.718841 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:01.718843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:01.719077 (+ 234us) server_negotiation.cc:407] Connection header received
0809 00:33:01.719718 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:01.719719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:01.719732 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:01.719784 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:01.720882 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:01.721393 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:01.722983 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:01.723191 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:01.723246 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:01.725674 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:01.725675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:01.725676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:01.725687 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:01.727174 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:01.727589 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:01.727590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:01.727591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:01.727649 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:01.727912 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:01.727913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:01.727913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:01.728030 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:01.728101 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:01.728409 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 00:33:01.728496 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:33:01.728790 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:01.718700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46507 (local address 127.21.41.193:50907)
0809 00:33:01.718937 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:01.718975 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:01.719592 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:01.719861 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:01.719865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:01.720266 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:01.720777 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:01.720783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:01.721496 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:01.721497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:01.722886 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:01.722890 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:01.723276 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:01.723277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:01.723429 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:01.723957 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:01.723968 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:01.725568 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:01.727279 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:01.727285 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:01.727287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:01.727509 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:01.727718 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:01.727719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:01.727719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:01.727840 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:01.728105 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:01.728112 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:01.728274 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 00:33:01.728387 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 00:33:01.728550 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":119,"threads_started":1}
I20260809 00:33:01.729321 25960 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46507
I20260809 00:33:01.729423 25960 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:01.729600 25960 heartbeater.cc:511] Master 127.21.41.254:46507 requested a full tablet report, sending...
I20260809 00:33:01.730150 25768 ts_manager.cc:194] Registered new tserver with Master: 643162250b9a4e4db02eb7793ff5a67c (127.21.41.193:39573)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
I20260809 00:33:01.731052 25768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:50907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:33:01.745282 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46507
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:01.826793 25965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:01.827000 25965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:01.827023 25965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:01.828133 25965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:01.828171 25965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:01.828195 25965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:33:01.829641 25965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:01.829876 25965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:01.830088 25965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:33:01.835147 25965 server_base.cc:1034] running on GCE node
W20260809 00:33:01.835156 25970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:01.835117 25971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:01.835117 25973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:01.835713 25965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:01.835929 25965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:01.837098 25965 hybrid_clock.cc:648] HybridClock initialized: now 1786235581837093 us; error 36 us; skew 500 ppm
Aug 09 00:33:01 dist-test-slave-zpfg krb5kdc[25723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:01.839763 25965 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:33:01.840106 25965 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:01.840154 25965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:01.840247 25965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:01.841269 25965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "d17fae862c124a3e932fa241b0812759"
format_stamp: "Formatted at 2026-08-09 00:33:01 on dist-test-slave-zpfg"
server_key: "e7335be495046d2695cf6f4ca0992ec3"
server_key_iv: "ea60ac878e2ee59f22181c7e47b0811b"
server_key_version: "encryptionkey@0"
I20260809 00:33:01.841576 25965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "d17fae862c124a3e932fa241b0812759"
format_stamp: "Formatted at 2026-08-09 00:33:01 on dist-test-slave-zpfg"
server_key: "e7335be495046d2695cf6f4ca0992ec3"
server_key_iv: "ea60ac878e2ee59f22181c7e47b0811b"
server_key_version: "encryptionkey@0"
I20260809 00:33:01.842895 25965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:33:01.843570 25979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:01.843757 25965 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:01.843851 25965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "d17fae862c124a3e932fa241b0812759"
format_stamp: "Formatted at 2026-08-09 00:33:01 on dist-test-slave-zpfg"
server_key: "e7335be495046d2695cf6f4ca0992ec3"
server_key_iv: "ea60ac878e2ee59f22181c7e47b0811b"
server_key_version: "encryptionkey@0"
I20260809 00:33:01.843927 25965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:01.852288 25965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:01.854029 25965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:01.854187 25965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:01.854465 25965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:01.854773 25965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:01.854806 25965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:01.854828 25965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:01.854874 25965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:01.861424 25965 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:38511
I20260809 00:33:01.861496 26093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:38511 every 8 connection(s)
I20260809 00:33:01.861758 25965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:33:01 dist-test-slave-zpfg krb5kdc[25723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:01.870539 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 25965
I20260809 00:33:01.870663 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:33:01.870891 21671 external_mini_cluster.cc:1428] Setting key cd1971cebf2e470cbfe545668ab304e9
I20260809 00:33:01.871415 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:01.862525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:46373 (local address 127.21.41.254:46507)
0809 00:33:01.862587 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:01.862589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:01.862812 (+ 223us) server_negotiation.cc:407] Connection header received
0809 00:33:01.863322 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:01.863324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:01.863336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:01.863400 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:01.864350 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:01.864856 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:01.865862 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:01.866080 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:01.866170 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:01.868516 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:01.868518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:01.868518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:01.868525 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:01.869781 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:01.870203 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:01.870205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:01.870205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:01.870270 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:01.870577 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:01.870578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:01.870579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:01.870739 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:01.870830 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:01.871224 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 00:33:01.871341 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:33:01.871606 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:01.862533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46507 (local address 127.21.41.194:46373)
0809 00:33:01.862693 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:01.862728 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:01.863217 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:01.863471 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:01.863475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:01.863736 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:01.864226 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:01.864232 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:01.864959 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:01.864960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:01.865763 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:01.865765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:01.866180 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:01.866181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:01.866322 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:01.866931 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:01.866941 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:01.868415 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:01.869866 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:01.869874 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:01.869876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:01.870099 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:01.870395 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:01.870396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:01.870396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:01.870506 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:01.870820 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:01.870827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:01.870995 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 00:33:01.871147 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 00:33:01.871376 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260809 00:33:01.872138 26094 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46507
I20260809 00:33:01.872232 26094 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:01.872400 26094 heartbeater.cc:511] Master 127.21.41.254:46507 requested a full tablet report, sending...
I20260809 00:33:01.872768 25768 ts_manager.cc:194] Registered new tserver with Master: d17fae862c124a3e932fa241b0812759 (127.21.41.194:38511)
I20260809 00:33:01.873150 25768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:46373
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:33:01.900987 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46507
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:01.992995 26099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:01.993211 26099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:01.993234 26099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:01.994519 26099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:01.994568 26099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:01.994596 26099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:01.996270 26099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:01.996518 26099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:01.996768 26099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:02.002069 26104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:02.002146 26099 server_base.cc:1034] running on GCE node
W20260809 00:33:02.002048 26107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:02.002048 26105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:02.002583 26099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:02.002835 26099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:02.003994 26099 hybrid_clock.cc:648] HybridClock initialized: now 1786235582003992 us; error 39 us; skew 500 ppm
Aug 09 00:33:01 dist-test-slave-zpfg krb5kdc[25723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:02.006577 26099 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:33:02.006953 26099 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:02.007027 26099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:02.007181 26099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:02.008317 26099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "f6538afa3e884394a8809b0df52244bf"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "834e50438666c74b0a044a12723d8940"
server_key_iv: "6347b3a8d583d216745f08df70896db3"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.008704 26099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "f6538afa3e884394a8809b0df52244bf"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "834e50438666c74b0a044a12723d8940"
server_key_iv: "6347b3a8d583d216745f08df70896db3"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.010241 26099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:33:02.011083 26113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.011251 26099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:02.011356 26099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "f6538afa3e884394a8809b0df52244bf"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "834e50438666c74b0a044a12723d8940"
server_key_iv: "6347b3a8d583d216745f08df70896db3"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.011440 26099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:02.022069 26099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:02.024374 26099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:02.024567 26099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:02.024852 26099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:02.025202 26099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:02.025262 26099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.025308 26099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:02.025332 26099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.031191 26099 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:44171
I20260809 00:33:02.031224 26226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:44171 every 8 connection(s)
I20260809 00:33:02.031517 26099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:33:02.036935 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 26099
I20260809 00:33:02.037053 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:33:02.037245 21671 external_mini_cluster.cc:1428] Setting key a9647a69ac4ced61202e60385817a36a
Aug 09 00:33:02 dist-test-slave-zpfg krb5kdc[25723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:02.041875 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.032277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:59215 (local address 127.21.41.254:46507)
0809 00:33:02.032345 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.032347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.032556 (+ 209us) server_negotiation.cc:407] Connection header received
0809 00:33:02.033111 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.033112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.033126 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.033175 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:02.034125 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.034708 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.035922 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.036270 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.036427 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.039085 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:02.039087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:02.039087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:02.039093 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:02.040693 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:02.041060 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:02.041061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:02.041061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:02.041117 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:02.041343 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:02.041344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:02.041344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:02.041451 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:02.041501 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.041739 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.041809 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:33:02.042078 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.032262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46507 (local address 127.21.41.195:59215)
0809 00:33:02.032441 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.032475 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.033005 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.033248 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.033252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.033526 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:02.034016 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.034021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.034806 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.034807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.035844 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.035847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.036366 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.036367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.036504 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.037183 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:02.037192 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:02.039016 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:02.040763 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:02.040768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:02.040769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:02.040982 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:02.041179 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:02.041179 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:02.041180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:02.041286 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:02.041504 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:02.041508 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:02.041617 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.041710 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.041835 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260809 00:33:02.042584 26227 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46507
I20260809 00:33:02.042693 26227 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:02.042852 26227 heartbeater.cc:511] Master 127.21.41.254:46507 requested a full tablet report, sending...
I20260809 00:33:02.043244 25768 ts_manager.cc:194] Registered new tserver with Master: f6538afa3e884394a8809b0df52244bf (127.21.41.195:44171)
I20260809 00:33:02.043583 25768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:59215
I20260809 00:33:02.049124 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:33:02.056157 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.050401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53554 (local address 127.21.41.254:46507)
0809 00:33:02.050450 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.050451 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.050539 (+ 88us) server_negotiation.cc:407] Connection header received
0809 00:33:02.050604 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.050605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.050613 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.050653 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:02.051199 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.051657 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.052338 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.052514 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.052565 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.053724 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:02.053725 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:02.053725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:02.053731 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:02.055204 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:02.055505 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:02.055506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:02.055506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:02.055563 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:02.055712 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:02.055713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:02.055713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:02.055852 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:02.055922 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.056012 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.056067 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:33:02.057960 25768 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:33:02.059641 25768 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:33:02.066315 26235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.061735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:39573 (local address 127.0.0.1:42004)
0809 00:33:02.061909 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.061948 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.062014 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.062578 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.062579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.062580 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:02.062620 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.062621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.063873 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.063874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.065118 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.065121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.066047 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.066048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.066140 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.066143 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.066174 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.066208 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":34,"thread_start_us":62,"threads_started":1}
I20260809 00:33:02.066308 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.062076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45652 (local address 127.21.41.194:38511)
0809 00:33:02.062321 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.062325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.062790 (+ 465us) server_negotiation.cc:407] Connection header received
0809 00:33:02.062834 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.062836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.062851 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.062922 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:02.063119 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.064222 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.065446 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.065911 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.066000 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.066060 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.066169 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.066204 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":94,"threads_started":1}
I20260809 00:33:02.066320 26236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.061744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:44171 (local address 127.0.0.1:49220)
0809 00:33:02.061984 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.061994 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.062048 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.062411 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.062414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.062417 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:02.062487 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.062489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.063940 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.063941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.064998 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.065001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.065924 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.065925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.066024 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.066041 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.066105 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.066181 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260809 00:33:02.066519 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.061745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:38511 (local address 127.0.0.1:45652)
0809 00:33:02.062520 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.062530 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.062581 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.063010 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.063010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.063012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:02.063049 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.063050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.064351 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.064352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.065362 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.065364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.066000 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.066002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.066058 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.066063 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.066155 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.066199 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":16,"thread_start_us":51,"threads_started":1}
I20260809 00:33:02.066962 26239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.061988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49220 (local address 127.21.41.195:44171)
0809 00:33:02.062186 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.062191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.062201 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:02.062244 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.062246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.062263 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.062386 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:02.062622 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.063828 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.065079 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.065821 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.066260 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.066333 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.066386 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.066427 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
I20260809 00:33:02.067045 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.061857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42004 (local address 127.21.41.193:39573)
0809 00:33:02.062177 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.062182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.062191 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:02.062241 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.062243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.062259 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.062393 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:02.062676 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.063776 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.065223 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.065959 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.066810 (+ 851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.066882 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.066908 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.066950 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":91,"threads_started":1}
I20260809 00:33:02.067409 26028 tablet_service.cc:1467] Processing CreateTablet for tablet 47467af7a50749c682df275028610f87 (DEFAULT_TABLE table=test-table [id=0f8ac044f6764009868cb3308d2096e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:02.067673 26161 tablet_service.cc:1467] Processing CreateTablet for tablet 47467af7a50749c682df275028610f87 (DEFAULT_TABLE table=test-table [id=0f8ac044f6764009868cb3308d2096e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:02.067800 26028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47467af7a50749c682df275028610f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:02.067778 25893 tablet_service.cc:1467] Processing CreateTablet for tablet 47467af7a50749c682df275028610f87 (DEFAULT_TABLE table=test-table [id=0f8ac044f6764009868cb3308d2096e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:02.068012 26161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47467af7a50749c682df275028610f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:02.068064 25893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47467af7a50749c682df275028610f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:02.070716 26241 tablet_bootstrap.cc:492] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: Bootstrap starting.
I20260809 00:33:02.071229 26242 tablet_bootstrap.cc:492] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: Bootstrap starting.
I20260809 00:33:02.071393 26241 tablet_bootstrap.cc:654] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:02.071527 26243 tablet_bootstrap.cc:492] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c: Bootstrap starting.
I20260809 00:33:02.071974 26242 tablet_bootstrap.cc:654] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:02.072111 26243 tablet_bootstrap.cc:654] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:02.072104 26241 log.cc:824] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:02.072554 26242 log.cc:824] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:02.072707 26243 log.cc:824] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:02.073014 26241 tablet_bootstrap.cc:492] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: No bootstrap required, opened a new log
I20260809 00:33:02.073089 26241 ts_tablet_manager.cc:1397] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:33:02.073477 26243 tablet_bootstrap.cc:492] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c: No bootstrap required, opened a new log
I20260809 00:33:02.073501 26242 tablet_bootstrap.cc:492] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: No bootstrap required, opened a new log
I20260809 00:33:02.073563 26242 ts_tablet_manager.cc:1397] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:33:02.073566 26243 ts_tablet_manager.cc:1397] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:33:02.074960 26243 raft_consensus.cc:348] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643162250b9a4e4db02eb7793ff5a67c" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39573 } } peers { permanent_uuid: "f6538afa3e884394a8809b0df52244bf" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44171 } } peers { permanent_uuid: "d17fae862c124a3e932fa241b0812759" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 38511 } }
I20260809 00:33:02.075117 26243 raft_consensus.cc:374] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:02.075178 26243 raft_consensus.cc:729] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643162250b9a4e4db02eb7793ff5a67c, State: Initialized, Role: FOLLOWER
I20260809 00:33:02.075158 26241 raft_consensus.cc:348] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643162250b9a4e4db02eb7793ff5a67c" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39573 } } peers { permanent_uuid: "f6538afa3e884394a8809b0df52244bf" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44171 } } peers { permanent_uuid: "d17fae862c124a3e932fa241b0812759" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 38511 } }
I20260809 00:33:02.075326 26241 raft_consensus.cc:374] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:02.075361 26243 consensus_queue.cc:260] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643162250b9a4e4db02eb7793ff5a67c" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39573 } } peers { permanent_uuid: "f6538afa3e884394a8809b0df52244bf" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44171 } } peers { permanent_uuid: "d17fae862c124a3e932fa241b0812759" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 38511 } }
I20260809 00:33:02.075410 26241 raft_consensus.cc:729] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d17fae862c124a3e932fa241b0812759, State: Initialized, Role: FOLLOWER
I20260809 00:33:02.075526 26241 consensus_queue.cc:260] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643162250b9a4e4db02eb7793ff5a67c" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39573 } } peers { permanent_uuid: "f6538afa3e884394a8809b0df52244bf" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44171 } } peers { permanent_uuid: "d17fae862c124a3e932fa241b0812759" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 38511 } }
I20260809 00:33:02.075670 25960 heartbeater.cc:503] Master 127.21.41.254:46507 was elected leader, sending a full tablet report...
I20260809 00:33:02.075667 26243 ts_tablet_manager.cc:1428] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:33:02.075624 26242 raft_consensus.cc:348] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643162250b9a4e4db02eb7793ff5a67c" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39573 } } peers { permanent_uuid: "f6538afa3e884394a8809b0df52244bf" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44171 } } peers { permanent_uuid: "d17fae862c124a3e932fa241b0812759" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 38511 } }
I20260809 00:33:02.075780 26242 raft_consensus.cc:374] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:02.075819 26242 raft_consensus.cc:729] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6538afa3e884394a8809b0df52244bf, State: Initialized, Role: FOLLOWER
I20260809 00:33:02.075912 26094 heartbeater.cc:503] Master 127.21.41.254:46507 was elected leader, sending a full tablet report...
I20260809 00:33:02.075918 26241 ts_tablet_manager.cc:1428] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:33:02.075925 26242 consensus_queue.cc:260] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643162250b9a4e4db02eb7793ff5a67c" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39573 } } peers { permanent_uuid: "f6538afa3e884394a8809b0df52244bf" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44171 } } peers { permanent_uuid: "d17fae862c124a3e932fa241b0812759" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 38511 } }
I20260809 00:33:02.076172 26242 ts_tablet_manager.cc:1428] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:33:02.076347 26227 heartbeater.cc:503] Master 127.21.41.254:46507 was elected leader, sending a full tablet report...
W20260809 00:33:02.112294 26095 tablet.cc:2369] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:33:02.218685 25961 tablet.cc:2369] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:33:02.282104 26228 tablet.cc:2369] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:33:02.301290 26248 raft_consensus.cc:482] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:33:02.301416 26248 raft_consensus.cc:504] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643162250b9a4e4db02eb7793ff5a67c" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39573 } } peers { permanent_uuid: "f6538afa3e884394a8809b0df52244bf" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44171 } } peers { permanent_uuid: "d17fae862c124a3e932fa241b0812759" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 38511 } }
I20260809 00:33:02.301836 26248 leader_election.cc:288] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643162250b9a4e4db02eb7793ff5a67c (127.21.41.193:39573), f6538afa3e884394a8809b0df52244bf (127.21.41.195:44171)
I20260809 00:33:02.304721 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.302084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:54041 (local address 127.21.41.193:39573)
0809 00:33:02.302139 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.302141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.302154 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:33:02.302291 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.302292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.302300 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.302414 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:02.302562 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.303063 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.304008 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.304365 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.304451 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.304460 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.304606 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.304643 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:33:02.304744 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.302012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:39573 (local address 127.21.41.194:54041)
0809 00:33:02.302105 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.302119 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.302168 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.302425 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.302426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.302428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:02.302487 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.302489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.303153 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.303154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.303904 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.303905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.304447 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.304448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.304520 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.304523 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.304598 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.304638 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 00:33:02.304770 26239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.302179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:44863 (local address 127.21.41.195:44171)
0809 00:33:02.302242 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.302244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.302312 (+ 68us) server_negotiation.cc:407] Connection header received
0809 00:33:02.302420 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.302421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.302431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.302505 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:02.302655 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.303126 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.304043 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.304417 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.304501 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.304508 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.304656 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.304690 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:33:02.305004 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.302050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:44171 (local address 127.21.41.194:44863)
0809 00:33:02.302226 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.302235 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.302321 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.302509 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.302510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.302511 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:02.302582 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.302584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.303214 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.303215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.303956 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:02.303957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.304501 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.304502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:02.304584 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.304587 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.304651 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.304689 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":103,"threads_started":1}
I20260809 00:33:02.305123 25913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47467af7a50749c682df275028610f87" candidate_uuid: "d17fae862c124a3e932fa241b0812759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643162250b9a4e4db02eb7793ff5a67c" is_pre_election: true
I20260809 00:33:02.305310 25913 raft_consensus.cc:2454] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d17fae862c124a3e932fa241b0812759 in term 0.
I20260809 00:33:02.305465 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47467af7a50749c682df275028610f87" candidate_uuid: "d17fae862c124a3e932fa241b0812759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6538afa3e884394a8809b0df52244bf" is_pre_election: true
I20260809 00:33:02.305562 25983 leader_election.cc:302] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643162250b9a4e4db02eb7793ff5a67c, d17fae862c124a3e932fa241b0812759; no voters:
I20260809 00:33:02.305631 26181 raft_consensus.cc:2454] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d17fae862c124a3e932fa241b0812759 in term 0.
I20260809 00:33:02.305717 26248 raft_consensus.cc:2781] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:33:02.305794 26248 raft_consensus.cc:482] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:33:02.305821 26248 raft_consensus.cc:3037] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:02.306521 26248 raft_consensus.cc:504] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643162250b9a4e4db02eb7793ff5a67c" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39573 } } peers { permanent_uuid: "f6538afa3e884394a8809b0df52244bf" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44171 } } peers { permanent_uuid: "d17fae862c124a3e932fa241b0812759" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 38511 } }
I20260809 00:33:02.306660 26248 leader_election.cc:288] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [CANDIDATE]: Term 1 election: Requested vote from peers 643162250b9a4e4db02eb7793ff5a67c (127.21.41.193:39573), f6538afa3e884394a8809b0df52244bf (127.21.41.195:44171)
I20260809 00:33:02.306831 26181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47467af7a50749c682df275028610f87" candidate_uuid: "d17fae862c124a3e932fa241b0812759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6538afa3e884394a8809b0df52244bf"
I20260809 00:33:02.306834 25913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47467af7a50749c682df275028610f87" candidate_uuid: "d17fae862c124a3e932fa241b0812759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643162250b9a4e4db02eb7793ff5a67c"
I20260809 00:33:02.306910 26181 raft_consensus.cc:3037] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:02.306929 25913 raft_consensus.cc:3037] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:02.307607 25913 raft_consensus.cc:2454] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d17fae862c124a3e932fa241b0812759 in term 1.
I20260809 00:33:02.307634 26181 raft_consensus.cc:2454] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d17fae862c124a3e932fa241b0812759 in term 1.
I20260809 00:33:02.307822 25980 leader_election.cc:302] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d17fae862c124a3e932fa241b0812759, f6538afa3e884394a8809b0df52244bf; no voters:
I20260809 00:33:02.307940 26248 raft_consensus.cc:2781] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:02.308106 26248 raft_consensus.cc:686] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 1 LEADER]: Becoming Leader. State: Replica: d17fae862c124a3e932fa241b0812759, State: Running, Role: LEADER
I20260809 00:33:02.308202 26248 consensus_queue.cc:237] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643162250b9a4e4db02eb7793ff5a67c" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39573 } } peers { permanent_uuid: "f6538afa3e884394a8809b0df52244bf" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44171 } } peers { permanent_uuid: "d17fae862c124a3e932fa241b0812759" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 38511 } }
I20260809 00:33:02.309041 25768 catalog_manager.cc:5495] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 reported cstate change: term changed from 0 to 1, leader changed from <none> to d17fae862c124a3e932fa241b0812759 (127.21.41.194). New cstate: current_term: 1 leader_uuid: "d17fae862c124a3e932fa241b0812759" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643162250b9a4e4db02eb7793ff5a67c" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6538afa3e884394a8809b0df52244bf" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d17fae862c124a3e932fa241b0812759" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 38511 } health_report { overall_health: HEALTHY } } }
I20260809 00:33:02.347432 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.344931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45666 (local address 127.21.41.194:38511)
0809 00:33:02.344992 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.344994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.345002 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:33:02.345049 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.345049 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.345067 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.345132 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:33:02.345297 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.345774 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.346711 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:02.346915 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:02.346980 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:02.347107 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:33:02.347238 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:33:02.347308 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.347342 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.347360 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:33:02.349256 25913 raft_consensus.cc:1261] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c [term 1 FOLLOWER]: Refusing update from remote peer d17fae862c124a3e932fa241b0812759: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:02.349264 26181 raft_consensus.cc:1261] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf [term 1 FOLLOWER]: Refusing update from remote peer d17fae862c124a3e932fa241b0812759: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:02.349680 26248 consensus_queue.cc:1035] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6538afa3e884394a8809b0df52244bf" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 44171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:02.349849 26251 consensus_queue.cc:1035] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643162250b9a4e4db02eb7793ff5a67c" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:02.352321 26254 mvcc.cc:205] Tried to move back new op lower bound from 7316420945299877888 to 7316420945135042560. Current Snapshot: MvccSnapshot[applied={T|T < 7316420945299877888}]
I20260809 00:33:02.352731 26256 mvcc.cc:205] Tried to move back new op lower bound from 7316420945299877888 to 7316420945135042560. Current Snapshot: MvccSnapshot[applied={T|T < 7316420945299877888}]
I20260809 00:33:02.353614 26255 mvcc.cc:205] Tried to move back new op lower bound from 7316420945299877888 to 7316420945135042560. Current Snapshot: MvccSnapshot[applied={T|T < 7316420945299877888 or (T in {7316420945299877888})}]
I20260809 00:33:02.354652 25768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:02.354729 25768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:02.355474 25768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 040ba4e93e964bd486b2df864e3c785f: Sending DeleteTablet for 3 replicas of tablet 47467af7a50749c682df275028610f87
I20260809 00:33:02.355815 25893 tablet_service.cc:1514] Processing DeleteTablet for tablet 47467af7a50749c682df275028610f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:02 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:42004
I20260809 00:33:02.355813 26161 tablet_service.cc:1514] Processing DeleteTablet for tablet 47467af7a50749c682df275028610f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:02 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:49220
I20260809 00:33:02.355834 26028 tablet_service.cc:1514] Processing DeleteTablet for tablet 47467af7a50749c682df275028610f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:02 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:45652
I20260809 00:33:02.356051 26266 tablet_replica.cc:330] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c: stopping tablet replica
I20260809 00:33:02.356094 26267 tablet_replica.cc:330] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: stopping tablet replica
I20260809 00:33:02.356160 26266 raft_consensus.cc:2229] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:02.356288 26266 raft_consensus.cc:2258] T 47467af7a50749c682df275028610f87 P 643162250b9a4e4db02eb7793ff5a67c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:02.356438 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 25831
I20260809 00:33:02.356444 26264 tablet_replica.cc:330] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: stopping tablet replica
I20260809 00:33:02.356529 26264 raft_consensus.cc:2229] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:02.356612 26264 raft_consensus.cc:2258] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:02.356993 26264 ts_tablet_manager.cc:1905] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:02.356230 26267 raft_consensus.cc:2229] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:33:02.359295 26264 ts_tablet_manager.cc:1918] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:02.359377 26264 log.cc:1196] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/47467af7a50749c682df275028610f87
I20260809 00:33:02.359633 26264 ts_tablet_manager.cc:1939] T 47467af7a50749c682df275028610f87 P f6538afa3e884394a8809b0df52244bf: Deleting consensus metadata
I20260809 00:33:02.360033 26267 raft_consensus.cc:2258] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:02.360039 25752 catalog_manager.cc:4841] TS f6538afa3e884394a8809b0df52244bf (127.21.41.195:44171): tablet 47467af7a50749c682df275028610f87 (table test-table [id=0f8ac044f6764009868cb3308d2096e5]) successfully deleted
I20260809 00:33:02.360507 26267 ts_tablet_manager.cc:1905] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:02.361913 26267 ts_tablet_manager.cc:1918] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:02.361994 26267 log.cc:1196] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/47467af7a50749c682df275028610f87
I20260809 00:33:02.362262 26267 ts_tablet_manager.cc:1939] T 47467af7a50749c682df275028610f87 P d17fae862c124a3e932fa241b0812759: Deleting consensus metadata
I20260809 00:33:02.362782 25754 catalog_manager.cc:4841] TS d17fae862c124a3e932fa241b0812759 (127.21.41.194:38511): tablet 47467af7a50749c682df275028610f87 (table test-table [id=0f8ac044f6764009868cb3308d2096e5]) successfully deleted
W20260809 00:33:02.364704 25755 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:39573 (error 108)
I20260809 00:33:02.365025 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 25965
I20260809 00:33:02.365181 26235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.365053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:39573 (local address 127.0.0.1:42014)
0809 00:33:02.365108 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.365144 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 00:33:02.365805 25755 catalog_manager.cc:4584] TS 643162250b9a4e4db02eb7793ff5a67c (127.21.41.193:39573): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47467af7a50749c682df275028610f87: Network error: Client connection negotiation failed: client connection to 127.21.41.193:39573: connect: Connection refused (error 111)
I20260809 00:33:02.369673 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 26099
I20260809 00:33:02.374289 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 25736
2026-08-09T00:33:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2126 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T00:33:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:02Z Disabled control of system clock
I20260809 00:33:02.402179 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:40135
--webserver_interface=127.21.41.254
--webserver_port=0
--builtin_ntp_servers=127.21.41.212:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:40135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:33:02.483047 26272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:02.483263 26272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:02.483285 26272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:02.484447 26272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:02.484496 26272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:02.484510 26272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:02.484522 26272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:02.484535 26272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:02.486064 26272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:40135
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:40135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:02.486353 26272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:02.486680 26272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:02.491428 26280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:02.491446 26278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:02.491432 26277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:02.491847 26272 server_base.cc:1034] running on GCE node
I20260809 00:33:02.492105 26272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:02.492408 26272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:02.493583 26272 hybrid_clock.cc:648] HybridClock initialized: now 1786235582493576 us; error 37 us; skew 500 ppm
I20260809 00:33:02.494853 26272 webserver.cc:460] Webserver started at http://127.21.41.254:35211/ using document root <none> and password file <none>
I20260809 00:33:02.495110 26272 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:02.495183 26272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:02.495354 26272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:02.496459 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "e2bbcb76f900440580e4534256982cab"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "b4e329cabd61b10f6b2ceff0b63aa5e2"
server_key_iv: "1f37f14c30c20543638cb7ce4b353dfb"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.496819 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "e2bbcb76f900440580e4534256982cab"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "b4e329cabd61b10f6b2ceff0b63aa5e2"
server_key_iv: "1f37f14c30c20543638cb7ce4b353dfb"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.498090 26272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:33:02.498878 26286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.499051 26272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.499150 26272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "e2bbcb76f900440580e4534256982cab"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "b4e329cabd61b10f6b2ceff0b63aa5e2"
server_key_iv: "1f37f14c30c20543638cb7ce4b353dfb"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.499239 26272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:02.502785 26272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:02.503221 26272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:02.503392 26272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:02.507684 26272 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:40135
I20260809 00:33:02.507710 26338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:40135 every 8 connection(s)
I20260809 00:33:02.508062 26272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:02.508694 26339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:02.511548 26339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab: Bootstrap starting.
I20260809 00:33:02.512094 26339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:02.512327 26339 log.cc:824] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:02.512974 26339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab: No bootstrap required, opened a new log
I20260809 00:33:02.514532 26339 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbcb76f900440580e4534256982cab" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40135 } }
I20260809 00:33:02.514667 26339 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:02.514689 26339 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2bbcb76f900440580e4534256982cab, State: Initialized, Role: FOLLOWER
I20260809 00:33:02.514834 26339 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbcb76f900440580e4534256982cab" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40135 } }
I20260809 00:33:02.514912 26339 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:02.514935 26339 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:02.514961 26339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:02.515429 26339 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbcb76f900440580e4534256982cab" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40135 } }
I20260809 00:33:02.515506 26339 leader_election.cc:302] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2bbcb76f900440580e4534256982cab; no voters:
I20260809 00:33:02.515637 26339 leader_election.cc:288] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:02.515708 26342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:02.515859 26342 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [term 1 LEADER]: Becoming Leader. State: Replica: e2bbcb76f900440580e4534256982cab, State: Running, Role: LEADER
I20260809 00:33:02.515904 26339 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:02.515944 26342 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbcb76f900440580e4534256982cab" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40135 } }
I20260809 00:33:02.516296 26343 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2bbcb76f900440580e4534256982cab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbcb76f900440580e4534256982cab" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40135 } } }
I20260809 00:33:02.516420 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 26272
I20260809 00:33:02.516451 26343 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:02.516541 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:02.516530 26344 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2bbcb76f900440580e4534256982cab. Latest consensus state: current_term: 1 leader_uuid: "e2bbcb76f900440580e4534256982cab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbcb76f900440580e4534256982cab" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40135 } } }
I20260809 00:33:02.516592 26344 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:02.516717 26350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:02.516788 21671 external_mini_cluster.cc:1428] Setting key 9ec903e0974b9b254106c5da9c108fc8
I20260809 00:33:02.517176 26350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:02.519026 26350 catalog_manager.cc:1347] Generated new cluster ID: 46869a87a36048c5ad4bdcf934c88a38
I20260809 00:33:02.519081 26350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:33:02.519977 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.517494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37408 (local address 127.21.41.254:40135)
0809 00:33:02.518243 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.518251 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.518497 (+ 246us) server_negotiation.cc:407] Connection header received
0809 00:33:02.518619 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.518627 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.518671 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.518822 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:02.519477 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:02.519481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:02.519481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:02.519485 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:02.519526 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:02.519545 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:02.519616 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.519781 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.519839 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":75,"threads_started":1}
I20260809 00:33:02.536244 26350 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:02.536450 26350 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:02.541134 26350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab: Generated new TSK 0
I20260809 00:33:02.541355 26350 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:33:02.550102 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40135
--builtin_ntp_servers=127.21.41.212:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:33:02.632179 26363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:02.632385 26363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:02.632407 26363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:02.633564 26363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:02.633601 26363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:02.633627 26363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:33:02.634989 26363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:02.635245 26363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:02.635592 26363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:02.640506 26368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:02.640602 26371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:02.640504 26369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:02.640861 26363 server_base.cc:1034] running on GCE node
I20260809 00:33:02.641058 26363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:02.641301 26363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:02.642460 26363 hybrid_clock.cc:648] HybridClock initialized: now 1786235582642442 us; error 32 us; skew 500 ppm
I20260809 00:33:02.643733 26363 webserver.cc:460] Webserver started at http://127.21.41.193:44507/ using document root <none> and password file <none>
I20260809 00:33:02.643987 26363 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:02.644058 26363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:02.644198 26363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:02.645289 26363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "cc4d94ee393a43e8b42301732043e659"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "34b46725752e16b9de94adb4419b30e0"
server_key_iv: "87c3a89eb197d92f3df9bfc7c9917515"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.645654 26363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "cc4d94ee393a43e8b42301732043e659"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "34b46725752e16b9de94adb4419b30e0"
server_key_iv: "87c3a89eb197d92f3df9bfc7c9917515"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.647101 26363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:33:02.647909 26377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.648056 26363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:02.648161 26363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "cc4d94ee393a43e8b42301732043e659"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "34b46725752e16b9de94adb4419b30e0"
server_key_iv: "87c3a89eb197d92f3df9bfc7c9917515"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.648237 26363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:02.651548 26363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:02.651822 26363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:02.651963 26363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:02.652171 26363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:02.652469 26363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:02.652503 26363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.652554 26363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:02.652580 26363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.658245 26363 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:38219
I20260809 00:33:02.658318 26490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:38219 every 8 connection(s)
I20260809 00:33:02.658650 26363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:33:02.661445 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.659404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:51419 (local address 127.21.41.254:40135)
0809 00:33:02.659486 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.659488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.659681 (+ 193us) server_negotiation.cc:407] Connection header received
0809 00:33:02.660116 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.660117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.660124 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.660175 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:02.661243 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:02.661244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:02.661244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:02.661248 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:02.661264 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:02.661268 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:02.661312 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.661378 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.661392 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:33:02.661643 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.659367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:40135 (local address 127.21.41.193:51419)
0809 00:33:02.659561 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.659594 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.660037 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.660232 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.660235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.660243 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:02.661106 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:02.661106 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:02.661133 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:02.661159 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:02.661316 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:02.661319 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:02.661321 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.661372 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.661418 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260809 00:33:02.662104 26491 heartbeater.cc:347] Connected to a master server at 127.21.41.254:40135
I20260809 00:33:02.662180 26491 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:02.662295 26491 heartbeater.cc:511] Master 127.21.41.254:40135 requested a full tablet report, sending...
I20260809 00:33:02.662773 26303 ts_manager.cc:194] Registered new tserver with Master: cc4d94ee393a43e8b42301732043e659 (127.21.41.193:38219)
I20260809 00:33:02.665306 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 26363
I20260809 00:33:02.665380 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:33:02.665589 21671 external_mini_cluster.cc:1428] Setting key 1e9e4d0f5f043c93f4be879e6bb11aca
I20260809 00:33:02.666517 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40135
--builtin_ntp_servers=127.21.41.212:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:33:02.746989 26494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:02.747197 26494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:02.747228 26494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:02.748462 26494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:02.748512 26494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:02.748544 26494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:33:02.750070 26494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:02.750312 26494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:02.750686 26494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:02.755718 26499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:02.755818 26494 server_base.cc:1034] running on GCE node
W20260809 00:33:02.755731 26500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:02.755726 26502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:02.756134 26494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:02.756394 26494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:02.757555 26494 hybrid_clock.cc:648] HybridClock initialized: now 1786235582757542 us; error 30 us; skew 500 ppm
I20260809 00:33:02.758824 26494 webserver.cc:460] Webserver started at http://127.21.41.194:43015/ using document root <none> and password file <none>
I20260809 00:33:02.759070 26494 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:02.759140 26494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:02.759301 26494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:02.760376 26494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "9277f68712d94d45ad4acf0f085dd532"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "c2c74cbb0295929fc1dfe61866b5e0ab"
server_key_iv: "09fb8b9a73e1eff2f84a2749f4d1f562"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.760737 26494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "9277f68712d94d45ad4acf0f085dd532"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "c2c74cbb0295929fc1dfe61866b5e0ab"
server_key_iv: "09fb8b9a73e1eff2f84a2749f4d1f562"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.762020 26494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:33:02.762770 26508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.763068 26494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:33:02.763170 26494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "9277f68712d94d45ad4acf0f085dd532"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "c2c74cbb0295929fc1dfe61866b5e0ab"
server_key_iv: "09fb8b9a73e1eff2f84a2749f4d1f562"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.763257 26494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:02.766536 26494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:02.766796 26494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:02.766937 26494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:02.767144 26494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:02.767444 26494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:02.767495 26494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.767534 26494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:02.767561 26494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.773389 26494 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:34953
I20260809 00:33:02.773571 26621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:34953 every 8 connection(s)
I20260809 00:33:02.773808 26494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:33:02.777426 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.774602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:38957 (local address 127.21.41.254:40135)
0809 00:33:02.774647 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.774649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.774993 (+ 344us) server_negotiation.cc:407] Connection header received
0809 00:33:02.775732 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.775733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.775741 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.775825 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:02.777072 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:02.777073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:02.777073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:02.777079 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:02.777098 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:02.777121 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:02.777193 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.777308 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.777326 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:33:02.777623 26624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.774599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:40135 (local address 127.21.41.194:38957)
0809 00:33:02.774804 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.774866 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.775605 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.775935 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.775940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.775949 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:02.776912 (+ 963us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:02.776912 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:02.776939 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:02.776965 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:02.777189 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:02.777229 (+ 40us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:02.777232 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.777326 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.777387 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260809 00:33:02.778123 26622 heartbeater.cc:347] Connected to a master server at 127.21.41.254:40135
I20260809 00:33:02.778224 26622 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:02.778462 26622 heartbeater.cc:511] Master 127.21.41.254:40135 requested a full tablet report, sending...
I20260809 00:33:02.778932 26303 ts_manager.cc:194] Registered new tserver with Master: 9277f68712d94d45ad4acf0f085dd532 (127.21.41.194:34953)
I20260809 00:33:02.781376 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 26494
I20260809 00:33:02.781478 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:33:02.781665 21671 external_mini_cluster.cc:1428] Setting key e8ed669128bfb8b5ebf5cc324c9fca81
I20260809 00:33:02.782721 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40135
--builtin_ntp_servers=127.21.41.212:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:33:02.863950 26625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:02.864158 26625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:02.864181 26625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:02.865288 26625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:02.865322 26625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:02.865347 26625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:02.866894 26625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:02.867172 26625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:02.867529 26625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:02.872792 26633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:02.872785 26631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:02.872784 26630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:02.873003 26625 server_base.cc:1034] running on GCE node
I20260809 00:33:02.873366 26625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:02.873643 26625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:02.874822 26625 hybrid_clock.cc:648] HybridClock initialized: now 1786235582874809 us; error 25 us; skew 500 ppm
I20260809 00:33:02.880359 26625 webserver.cc:460] Webserver started at http://127.21.41.195:34061/ using document root <none> and password file <none>
I20260809 00:33:02.880628 26625 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:02.880700 26625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:02.880857 26625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:02.881973 26625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "c48218e359394e8ba76f7765f3508210"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "2364a10b4f9ff1a0e438b1103d7f25ee"
server_key_iv: "511c7845f57220e670828d8386ec9347"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.882391 26625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "c48218e359394e8ba76f7765f3508210"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "2364a10b4f9ff1a0e438b1103d7f25ee"
server_key_iv: "511c7845f57220e670828d8386ec9347"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.883842 26625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:02.884617 26639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.884840 26625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:02.884949 26625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "c48218e359394e8ba76f7765f3508210"
format_stamp: "Formatted at 2026-08-09 00:33:02 on dist-test-slave-zpfg"
server_key: "2364a10b4f9ff1a0e438b1103d7f25ee"
server_key_iv: "511c7845f57220e670828d8386ec9347"
server_key_version: "encryptionkey@0"
I20260809 00:33:02.885035 26625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:02.888495 26625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:02.888783 26625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:02.888926 26625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:02.889130 26625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:02.889438 26625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:02.889492 26625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.889524 26625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:02.889551 26625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:02.895296 26625 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:43137
I20260809 00:33:02.895344 26754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:43137 every 8 connection(s)
I20260809 00:33:02.895678 26625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:33:02.897588 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 26625
I20260809 00:33:02.897693 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:33:02.897915 21671 external_mini_cluster.cc:1428] Setting key 094e8b2165b5db8ace129b3a17550fc4
I20260809 00:33:02.898660 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.896368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:49011 (local address 127.21.41.254:40135)
0809 00:33:02.896425 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.896428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.896629 (+ 201us) server_negotiation.cc:407] Connection header received
0809 00:33:02.897062 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.897063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.897066 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.897126 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:02.898394 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:02.898396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:02.898396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:02.898402 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:02.898419 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:02.898424 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:02.898473 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.898572 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.898588 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 00:33:02.898877 26757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.896326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:40135 (local address 127.21.41.195:49011)
0809 00:33:02.896506 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.896546 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.896984 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.897186 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.897189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.897196 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:02.898209 (+ 1013us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:02.898209 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:02.898237 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:02.898269 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:02.898482 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:02.898487 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:02.898490 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.898582 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.898642 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260809 00:33:02.899320 26755 heartbeater.cc:347] Connected to a master server at 127.21.41.254:40135
I20260809 00:33:02.899420 26755 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:02.899564 26755 heartbeater.cc:511] Master 127.21.41.254:40135 requested a full tablet report, sending...
I20260809 00:33:02.899935 26303 ts_manager.cc:194] Registered new tserver with Master: c48218e359394e8ba76f7765f3508210 (127.21.41.195:43137)
I20260809 00:33:02.909863 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:33:02.912472 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.911309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.21.41.254:40135)
0809 00:33:02.911355 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.911357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.911530 (+ 173us) server_negotiation.cc:407] Connection header received
0809 00:33:02.911576 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.911576 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.911581 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.911623 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:02.912265 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:02.912266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:02.912266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:02.912269 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:02.912282 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:02.912285 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:02.912325 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.912355 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.912367 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:33:02.914055 26303 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:33:02.915755 26303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:33:02.919919 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.918315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58836 (local address 127.21.41.195:43137)
0809 00:33:02.918549 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.918554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.918621 (+ 67us) server_negotiation.cc:407] Connection header received
0809 00:33:02.918675 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.918677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.918688 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.918774 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:02.919567 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:02.919575 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:02.919575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:02.919579 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:02.919615 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:02.919644 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:02.919727 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.919809 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.919824 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":116,"threads_started":1}
I20260809 00:33:02.919929 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.918032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:34953 (local address 127.0.0.1:52242)
0809 00:33:02.918427 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.918438 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.918471 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.918790 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.918796 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.918802 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:02.919338 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:02.919338 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:02.919354 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:02.919379 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:02.919731 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:02.919732 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:02.919733 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.919810 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.919824 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":103,"threads_started":1}
I20260809 00:33:02.919929 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.918042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:43137 (local address 127.0.0.1:58836)
0809 00:33:02.918558 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.918567 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.918615 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.919023 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.919024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.919026 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:02.919476 (+ 450us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:02.919476 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:02.919482 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:02.919495 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:02.919720 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:02.919728 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:02.919731 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.919793 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.919807 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":105,"threads_started":1}
I20260809 00:33:02.920222 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.918445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52242 (local address 127.21.41.194:34953)
0809 00:33:02.918638 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.918641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.918652 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:33:02.918678 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.918680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.918688 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.918788 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:02.919578 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:02.919588 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:02.919588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:02.919593 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:02.919627 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:02.919647 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:02.919704 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.920128 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.920147 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":82,"threads_started":1}
I20260809 00:33:02.920286 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.918031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38219 (local address 127.0.0.1:51594)
0809 00:33:02.918216 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:02.918259 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:02.918315 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:02.918905 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:02.918906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:02.918908 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:02.919885 (+ 977us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:02.919885 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:02.919891 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:02.919901 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:02.920153 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:02.920153 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:02.920154 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:33:02.920202 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 00:33:02.920230 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260809 00:33:02.920552 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:02.918217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.21.41.193:38219)
0809 00:33:02.918440 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:02.918445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:02.918457 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:33:02.918487 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:02.918490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:02.918501 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:02.918580 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:02.919989 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:02.919999 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:02.919999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:02.920003 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:02.920037 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:02.920064 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:02.920124 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:02.920425 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 00:33:02.920442 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":113,"threads_started":1}
I20260809 00:33:02.920678 26688 tablet_service.cc:1467] Processing CreateTablet for tablet e152ed4b8af448bba6f3c31b3578937c (DEFAULT_TABLE table=test-table [id=0d5bf29788554ee09419ead69e8c109e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:02.920729 26556 tablet_service.cc:1467] Processing CreateTablet for tablet e152ed4b8af448bba6f3c31b3578937c (DEFAULT_TABLE table=test-table [id=0d5bf29788554ee09419ead69e8c109e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:02.921042 26556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e152ed4b8af448bba6f3c31b3578937c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:02.921042 26688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e152ed4b8af448bba6f3c31b3578937c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:02.921241 26425 tablet_service.cc:1467] Processing CreateTablet for tablet e152ed4b8af448bba6f3c31b3578937c (DEFAULT_TABLE table=test-table [id=0d5bf29788554ee09419ead69e8c109e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:02.921605 26425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e152ed4b8af448bba6f3c31b3578937c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:02.924468 26769 tablet_bootstrap.cc:492] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659: Bootstrap starting.
I20260809 00:33:02.924674 26770 tablet_bootstrap.cc:492] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: Bootstrap starting.
I20260809 00:33:02.925132 26769 tablet_bootstrap.cc:654] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:02.925280 26770 tablet_bootstrap.cc:654] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:02.925468 26769 log.cc:824] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:02.925601 26770 log.cc:824] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:02.925944 26771 tablet_bootstrap.cc:492] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: Bootstrap starting.
I20260809 00:33:02.926326 26769 tablet_bootstrap.cc:492] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659: No bootstrap required, opened a new log
I20260809 00:33:02.926388 26770 tablet_bootstrap.cc:492] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: No bootstrap required, opened a new log
I20260809 00:33:02.926437 26770 ts_tablet_manager.cc:1397] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:33:02.926433 26769 ts_tablet_manager.cc:1397] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 00:33:02.926633 26771 tablet_bootstrap.cc:654] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:02.926920 26771 log.cc:824] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:02.927605 26771 tablet_bootstrap.cc:492] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: No bootstrap required, opened a new log
I20260809 00:33:02.927682 26771 ts_tablet_manager.cc:1397] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:33:02.929118 26770 raft_consensus.cc:348] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9277f68712d94d45ad4acf0f085dd532" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34953 } } peers { permanent_uuid: "cc4d94ee393a43e8b42301732043e659" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38219 } } peers { permanent_uuid: "c48218e359394e8ba76f7765f3508210" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43137 } }
I20260809 00:33:02.929118 26769 raft_consensus.cc:348] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9277f68712d94d45ad4acf0f085dd532" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34953 } } peers { permanent_uuid: "cc4d94ee393a43e8b42301732043e659" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38219 } } peers { permanent_uuid: "c48218e359394e8ba76f7765f3508210" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43137 } }
I20260809 00:33:02.929294 26770 raft_consensus.cc:374] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:02.929294 26769 raft_consensus.cc:374] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:02.929333 26769 raft_consensus.cc:729] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4d94ee393a43e8b42301732043e659, State: Initialized, Role: FOLLOWER
I20260809 00:33:02.929333 26770 raft_consensus.cc:729] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9277f68712d94d45ad4acf0f085dd532, State: Initialized, Role: FOLLOWER
I20260809 00:33:02.929461 26770 consensus_queue.cc:260] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9277f68712d94d45ad4acf0f085dd532" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34953 } } peers { permanent_uuid: "cc4d94ee393a43e8b42301732043e659" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38219 } } peers { permanent_uuid: "c48218e359394e8ba76f7765f3508210" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43137 } }
I20260809 00:33:02.929489 26769 consensus_queue.cc:260] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9277f68712d94d45ad4acf0f085dd532" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34953 } } peers { permanent_uuid: "cc4d94ee393a43e8b42301732043e659" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38219 } } peers { permanent_uuid: "c48218e359394e8ba76f7765f3508210" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43137 } }
I20260809 00:33:02.929569 26771 raft_consensus.cc:348] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9277f68712d94d45ad4acf0f085dd532" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34953 } } peers { permanent_uuid: "cc4d94ee393a43e8b42301732043e659" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38219 } } peers { permanent_uuid: "c48218e359394e8ba76f7765f3508210" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43137 } }
I20260809 00:33:02.929725 26769 ts_tablet_manager.cc:1428] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:33:02.929735 26770 ts_tablet_manager.cc:1428] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:33:02.929725 26771 raft_consensus.cc:374] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:02.929774 26771 raft_consensus.cc:729] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48218e359394e8ba76f7765f3508210, State: Initialized, Role: FOLLOWER
I20260809 00:33:02.929841 26491 heartbeater.cc:503] Master 127.21.41.254:40135 was elected leader, sending a full tablet report...
I20260809 00:33:02.929903 26622 heartbeater.cc:503] Master 127.21.41.254:40135 was elected leader, sending a full tablet report...
I20260809 00:33:02.929888 26771 consensus_queue.cc:260] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9277f68712d94d45ad4acf0f085dd532" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34953 } } peers { permanent_uuid: "cc4d94ee393a43e8b42301732043e659" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38219 } } peers { permanent_uuid: "c48218e359394e8ba76f7765f3508210" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43137 } }
I20260809 00:33:02.930130 26771 ts_tablet_manager.cc:1428] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 00:33:02.930475 26755 heartbeater.cc:503] Master 127.21.41.254:40135 was elected leader, sending a full tablet report...
W20260809 00:33:03.026113 26623 tablet.cc:2369] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:33:03.146245 26756 tablet.cc:2369] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:33:03.158921 26776 raft_consensus.cc:482] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:33:03.159046 26776 raft_consensus.cc:504] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9277f68712d94d45ad4acf0f085dd532" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34953 } } peers { permanent_uuid: "cc4d94ee393a43e8b42301732043e659" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38219 } } peers { permanent_uuid: "c48218e359394e8ba76f7765f3508210" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43137 } }
W20260809 00:33:03.159351 26492 tablet.cc:2369] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:33:03.159564 26776 leader_election.cc:288] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc4d94ee393a43e8b42301732043e659 (127.21.41.193:38219), c48218e359394e8ba76f7765f3508210 (127.21.41.195:43137)
I20260809 00:33:03.160936 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.159860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:40641 (local address 127.21.41.193:38219)
0809 00:33:03.159908 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:03.159910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:03.159920 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:03.159989 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:03.159989 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:03.159994 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:03.160041 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:03.160664 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:03.160665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:03.160666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:03.160670 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:03.160689 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:03.160695 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:03.160759 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:03.160856 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 00:33:03.160873 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:33:03.160936 26624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.159773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38219 (local address 127.21.41.194:40641)
0809 00:33:03.159857 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:03.159869 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:03.159917 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:03.160045 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:03.160045 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:03.160048 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:03.160551 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:03.160551 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:03.160560 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:03.160575 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:03.160778 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:03.160779 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:03.160780 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:33:03.160850 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:33:03.160867 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:33:03.161036 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.159892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:53221 (local address 127.21.41.195:43137)
0809 00:33:03.159943 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:03.159945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:03.160084 (+ 139us) server_negotiation.cc:407] Connection header received
0809 00:33:03.160158 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:03.160159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:03.160164 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:03.160244 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:03.160828 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:03.160830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:03.160830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:03.160834 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:03.160849 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:03.160853 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:03.160904 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:03.160956 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 00:33:03.160970 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:33:03.161048 26778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.159776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:43137 (local address 127.21.41.194:53221)
0809 00:33:03.160004 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:03.160023 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:03.160093 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:03.160253 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:03.160254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:03.160256 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:03.160746 (+ 490us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:03.160747 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:03.160754 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:03.160767 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:03.160901 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:03.160901 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:03.160902 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:33:03.160955 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:33:03.160970 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":66,"thread_start_us":104,"threads_started":1}
I20260809 00:33:03.161296 26445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e152ed4b8af448bba6f3c31b3578937c" candidate_uuid: "9277f68712d94d45ad4acf0f085dd532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4d94ee393a43e8b42301732043e659" is_pre_election: true
I20260809 00:33:03.161477 26445 raft_consensus.cc:2454] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9277f68712d94d45ad4acf0f085dd532 in term 0.
I20260809 00:33:03.161465 26708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e152ed4b8af448bba6f3c31b3578937c" candidate_uuid: "9277f68712d94d45ad4acf0f085dd532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48218e359394e8ba76f7765f3508210" is_pre_election: true
I20260809 00:33:03.161586 26708 raft_consensus.cc:2454] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9277f68712d94d45ad4acf0f085dd532 in term 0.
I20260809 00:33:03.161715 26511 leader_election.cc:302] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9277f68712d94d45ad4acf0f085dd532, cc4d94ee393a43e8b42301732043e659; no voters:
I20260809 00:33:03.161958 26776 raft_consensus.cc:2781] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:33:03.162034 26776 raft_consensus.cc:482] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:33:03.162072 26776 raft_consensus.cc:3037] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:03.162870 26776 raft_consensus.cc:504] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9277f68712d94d45ad4acf0f085dd532" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34953 } } peers { permanent_uuid: "cc4d94ee393a43e8b42301732043e659" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38219 } } peers { permanent_uuid: "c48218e359394e8ba76f7765f3508210" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43137 } }
I20260809 00:33:03.163024 26776 leader_election.cc:288] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [CANDIDATE]: Term 1 election: Requested vote from peers cc4d94ee393a43e8b42301732043e659 (127.21.41.193:38219), c48218e359394e8ba76f7765f3508210 (127.21.41.195:43137)
I20260809 00:33:03.163166 26445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e152ed4b8af448bba6f3c31b3578937c" candidate_uuid: "9277f68712d94d45ad4acf0f085dd532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4d94ee393a43e8b42301732043e659"
I20260809 00:33:03.163221 26708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e152ed4b8af448bba6f3c31b3578937c" candidate_uuid: "9277f68712d94d45ad4acf0f085dd532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48218e359394e8ba76f7765f3508210"
I20260809 00:33:03.163272 26445 raft_consensus.cc:3037] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:03.163285 26708 raft_consensus.cc:3037] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:03.164166 26708 raft_consensus.cc:2454] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9277f68712d94d45ad4acf0f085dd532 in term 1.
I20260809 00:33:03.164166 26445 raft_consensus.cc:2454] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9277f68712d94d45ad4acf0f085dd532 in term 1.
I20260809 00:33:03.164377 26509 leader_election.cc:302] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9277f68712d94d45ad4acf0f085dd532, c48218e359394e8ba76f7765f3508210; no voters:
I20260809 00:33:03.164466 26776 raft_consensus.cc:2781] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:03.164615 26776 raft_consensus.cc:686] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 1 LEADER]: Becoming Leader. State: Replica: 9277f68712d94d45ad4acf0f085dd532, State: Running, Role: LEADER
I20260809 00:33:03.164731 26776 consensus_queue.cc:237] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9277f68712d94d45ad4acf0f085dd532" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34953 } } peers { permanent_uuid: "cc4d94ee393a43e8b42301732043e659" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38219 } } peers { permanent_uuid: "c48218e359394e8ba76f7765f3508210" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43137 } }
I20260809 00:33:03.165648 26303 catalog_manager.cc:5495] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9277f68712d94d45ad4acf0f085dd532 (127.21.41.194). New cstate: current_term: 1 leader_uuid: "9277f68712d94d45ad4acf0f085dd532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9277f68712d94d45ad4acf0f085dd532" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4d94ee393a43e8b42301732043e659" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48218e359394e8ba76f7765f3508210" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43137 } health_report { overall_health: UNKNOWN } } }
I20260809 00:33:03.203835 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.202716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.21.41.194:34953)
0809 00:33:03.202801 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:03.202803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:03.202813 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:03.202861 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:03.202861 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:03.202875 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:03.202942 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:03.203573 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:03.203574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:03.203574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:03.203579 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:03.203603 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:03.203608 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:03.203671 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:03.203737 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 00:33:03.203752 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 00:33:03.205744 26708 raft_consensus.cc:1261] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210 [term 1 FOLLOWER]: Refusing update from remote peer 9277f68712d94d45ad4acf0f085dd532: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:03.205758 26445 raft_consensus.cc:1261] T e152ed4b8af448bba6f3c31b3578937c P cc4d94ee393a43e8b42301732043e659 [term 1 FOLLOWER]: Refusing update from remote peer 9277f68712d94d45ad4acf0f085dd532: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:03.206130 26779 consensus_queue.cc:1035] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4d94ee393a43e8b42301732043e659" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:03.206308 26776 consensus_queue.cc:1035] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48218e359394e8ba76f7765f3508210" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:03.211808 26303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:03.211915 26303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:03.212684 26303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2bbcb76f900440580e4534256982cab: Sending DeleteTablet for 3 replicas of tablet e152ed4b8af448bba6f3c31b3578937c
I20260809 00:33:03.213068 26688 tablet_service.cc:1514] Processing DeleteTablet for tablet e152ed4b8af448bba6f3c31b3578937c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:03 UTC) from {username='slave'} at 127.0.0.1:58836
I20260809 00:33:03.213116 26556 tablet_service.cc:1514] Processing DeleteTablet for tablet e152ed4b8af448bba6f3c31b3578937c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:03 UTC) from {username='slave'} at 127.0.0.1:52242
I20260809 00:33:03.213421 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 26363
I20260809 00:33:03.213433 26425 tablet_service.cc:1514] Processing DeleteTablet for tablet e152ed4b8af448bba6f3c31b3578937c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:03 UTC) from {username='slave'} at 127.0.0.1:51594
I20260809 00:33:03.214303 26792 tablet_replica.cc:330] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: stopping tablet replica
I20260809 00:33:03.214432 26792 raft_consensus.cc:2229] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:33:03.214557 26792 raft_consensus.cc:2258] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:03.214980 26792 ts_tablet_manager.cc:1905] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:03.216615 26791 tablet_replica.cc:330] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: stopping tablet replica
I20260809 00:33:03.216748 26791 raft_consensus.cc:2229] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:03.216879 26791 raft_consensus.cc:2258] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:03.217067 26792 ts_tablet_manager.cc:1918] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:03.217144 26792 log.cc:1196] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/e152ed4b8af448bba6f3c31b3578937c
I20260809 00:33:03.217469 26792 ts_tablet_manager.cc:1939] T e152ed4b8af448bba6f3c31b3578937c P 9277f68712d94d45ad4acf0f085dd532: Deleting consensus metadata
I20260809 00:33:03.217581 26791 ts_tablet_manager.cc:1905] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:03.217856 26288 catalog_manager.cc:4841] TS 9277f68712d94d45ad4acf0f085dd532 (127.21.41.194:34953): tablet e152ed4b8af448bba6f3c31b3578937c (table test-table [id=0d5bf29788554ee09419ead69e8c109e]) successfully deleted
I20260809 00:33:03.219208 26791 ts_tablet_manager.cc:1918] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:03.219297 26791 log.cc:1196] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/e152ed4b8af448bba6f3c31b3578937c
I20260809 00:33:03.219534 26791 ts_tablet_manager.cc:1939] T e152ed4b8af448bba6f3c31b3578937c P c48218e359394e8ba76f7765f3508210: Deleting consensus metadata
I20260809 00:33:03.219928 26287 catalog_manager.cc:4841] TS c48218e359394e8ba76f7765f3508210 (127.21.41.195:43137): tablet e152ed4b8af448bba6f3c31b3578937c (table test-table [id=0d5bf29788554ee09419ead69e8c109e]) successfully deleted
W20260809 00:33:03.220512 26289 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:38219 (error 108)
I20260809 00:33:03.220786 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 26494
I20260809 00:33:03.220959 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.220838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38219 (local address 127.0.0.1:51600)
0809 00:33:03.220888 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:03.220923 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 00:33:03.222580 26289 catalog_manager.cc:4584] TS cc4d94ee393a43e8b42301732043e659 (127.21.41.193:38219): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e152ed4b8af448bba6f3c31b3578937c: Network error: Client connection negotiation failed: client connection to 127.21.41.193:38219: connect: Connection refused (error 111)
I20260809 00:33:03.225859 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 26625
I20260809 00:33:03.230293 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 26272
2026-08-09T00:33:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (855 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 00:33:03.250190 21671 mini_oidc.cc:152] Starting JWKS server
I20260809 00:33:03.251523 21671 webserver.cc:460] Webserver started at https://127.0.0.1:32987/ using document root <none> and password file <none>
I20260809 00:33:03.251652 21671 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:33:03.251835 21671 webserver.cc:460] Webserver started at http://127.0.0.1:43127/ using document root <none> and password file <none>
2026-08-09T00:33:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:03Z Disabled control of system clock
I20260809 00:33:03.260756 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:39941
--webserver_interface=127.21.41.254
--webserver_port=0
--builtin_ntp_servers=127.21.41.212:41805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:39941
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32987/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/testchainca.cert with env {}
W20260809 00:33:03.345484 26799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:03.345701 26799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:03.345724 26799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:03.346894 26799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:03.346931 26799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:03.346944 26799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:03.346957 26799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:03.346969 26799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:33:03.346982 26799 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:33:03.346993 26799 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32987/jwks.json
I20260809 00:33:03.348543 26799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:41805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:39941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:39941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32987/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:03.348748 26799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:03.348917 26799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:03.353806 26807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:03.353811 26805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:03.353889 26804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:03.353946 26799 server_base.cc:1034] running on GCE node
I20260809 00:33:03.354282 26799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:03.354591 26799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:03.355782 26799 hybrid_clock.cc:648] HybridClock initialized: now 1786235583355759 us; error 34 us; skew 500 ppm
I20260809 00:33:03.356976 26799 webserver.cc:460] Webserver started at http://127.21.41.254:42331/ using document root <none> and password file <none>
I20260809 00:33:03.357254 26799 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:03.357326 26799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:03.357471 26799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:03.358603 26799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "262c9d6c74ac463da17a709c616dde1e"
format_stamp: "Formatted at 2026-08-09 00:33:03 on dist-test-slave-zpfg"
server_key: "2c46575669ba8931bc4c500084bfc338"
server_key_iv: "ae127aa52673512b75bfc81859d1e8a6"
server_key_version: "encryptionkey@0"
I20260809 00:33:03.358951 26799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "262c9d6c74ac463da17a709c616dde1e"
format_stamp: "Formatted at 2026-08-09 00:33:03 on dist-test-slave-zpfg"
server_key: "2c46575669ba8931bc4c500084bfc338"
server_key_iv: "ae127aa52673512b75bfc81859d1e8a6"
server_key_version: "encryptionkey@0"
I20260809 00:33:03.360268 26799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:33:03.360989 26813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:03.361152 26799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:03.361253 26799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "262c9d6c74ac463da17a709c616dde1e"
format_stamp: "Formatted at 2026-08-09 00:33:03 on dist-test-slave-zpfg"
server_key: "2c46575669ba8931bc4c500084bfc338"
server_key_iv: "ae127aa52673512b75bfc81859d1e8a6"
server_key_version: "encryptionkey@0"
I20260809 00:33:03.361334 26799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:03.388639 26799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:03.389009 26799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:03.389175 26799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:03.393462 26799 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:39941
I20260809 00:33:03.393512 26867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:39941 every 8 connection(s)
I20260809 00:33:03.393894 26799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:03.394990 26868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:03.395800 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 26799
I20260809 00:33:03.395913 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:03.396128 21671 external_mini_cluster.cc:1428] Setting key 066c7d7c4390a31b96667a2aae95e912
I20260809 00:33:03.398306 26868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e: Bootstrap starting.
I20260809 00:33:03.399008 26868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:03.399334 26868 log.cc:824] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:03.400094 26868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e: No bootstrap required, opened a new log
I20260809 00:33:03.401981 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.396689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52082 (local address 127.21.41.254:39941)
0809 00:33:03.396912 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:03.396920 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:03.396950 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:33:03.397417 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:03.397424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:03.397487 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:03.397603 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:03.397859 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.398887 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.399967 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.400464 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.400645 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:03.401307 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:03.401310 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:03.401311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:03.401481 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:03.401517 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:03.401528 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:03.401578 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:03.401685 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 00:33:03.401749 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":71,"threads_started":1}
I20260809 00:33:03.402477 26868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9d6c74ac463da17a709c616dde1e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39941 } }
I20260809 00:33:03.402658 26868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:03.402700 26868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262c9d6c74ac463da17a709c616dde1e, State: Initialized, Role: FOLLOWER
I20260809 00:33:03.402824 26868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9d6c74ac463da17a709c616dde1e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39941 } }
I20260809 00:33:03.402904 26868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:03.402946 26868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:03.402987 26868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:03.402985 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:33:03.403520 26868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9d6c74ac463da17a709c616dde1e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39941 } }
I20260809 00:33:03.403649 26868 leader_election.cc:302] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 262c9d6c74ac463da17a709c616dde1e; no voters:
I20260809 00:33:03.403831 26868 leader_election.cc:288] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:03.403887 26873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:03.404050 26873 raft_consensus.cc:686] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [term 1 LEADER]: Becoming Leader. State: Replica: 262c9d6c74ac463da17a709c616dde1e, State: Running, Role: LEADER
I20260809 00:33:03.404096 26868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:03.404139 26873 consensus_queue.cc:237] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9d6c74ac463da17a709c616dde1e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39941 } }
I20260809 00:33:03.404498 26873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 262c9d6c74ac463da17a709c616dde1e. Latest consensus state: current_term: 1 leader_uuid: "262c9d6c74ac463da17a709c616dde1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9d6c74ac463da17a709c616dde1e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39941 } } }
I20260809 00:33:03.404507 26874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "262c9d6c74ac463da17a709c616dde1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "262c9d6c74ac463da17a709c616dde1e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39941 } } }
I20260809 00:33:03.404582 26873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:03.404614 26874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e [sys.catalog]: This master's current role is: LEADER
W20260809 00:33:03.405850 26889 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:33:03.405915 26889 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:33:03.406010 26879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:03.406215 26879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:03.407757 26879 catalog_manager.cc:1347] Generated new cluster ID: 6fe5acbd9712438ba9142ba40e33d4bb
I20260809 00:33:03.407814 26879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:33:03.418210 26879 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:03.418982 26879 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:03.432271 26879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 262c9d6c74ac463da17a709c616dde1e: Generated new TSK 0
I20260809 00:33:03.432509 26879 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 00:33:03.457614 26898 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 00:33:03.457784 26898 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260809 00:33:03.458477 26897 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:03.441798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:39941 (local address 127.0.0.1:52092)
0809 00:33:03.442002 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:03.442035 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:03.442100 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:03.442323 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:03.442323 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:03.442329 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:33:03.442424 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:03.442426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.443390 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.443391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:03.443969 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:03.443970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.444455 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.444456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:03.444495 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:03.444514 (+ 19us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:33:03.457908 (+ 13394us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 00:33:03.458022 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.41.254:39941: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260809 00:33:03.458671 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.441916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52092 (local address 127.21.41.254:39941)
0809 00:33:03.441972 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:03.441975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:03.442112 (+ 137us) server_negotiation.cc:407] Connection header received
0809 00:33:03.442193 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:03.442194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:03.442208 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:03.442290 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:33:03.442501 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.443314 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.444051 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.444367 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.444440 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:03.444634 (+ 194us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:33:03.457731 (+ 13097us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 00:33:03.458537 (+ 806us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52092: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 00:33:03.458742 26871 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52092: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.41.254:39941: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 00:33:03.459090 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 26799
2026-08-09T00:33:03Z chronyd exiting
I20260809 00:33:03.495738 21671 test_util.cc:175] -----------------------------------------------
I20260809 00:33:03.495855 21671 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786235559853195-21671-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (246 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 00:33:03.496678 21671 mini_oidc.cc:152] Starting JWKS server
I20260809 00:33:03.497443 21671 webserver.cc:460] Webserver started at https://127.0.0.1:40851/ using document root <none> and password file <none>
I20260809 00:33:03.497622 21671 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:33:03.497887 21671 webserver.cc:460] Webserver started at http://127.0.0.1:42401/ using document root <none> and password file <none>
2026-08-09T00:33:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:03Z Disabled control of system clock
I20260809 00:33:03.507460 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:36317
--webserver_interface=127.21.41.254
--webserver_port=0
--builtin_ntp_servers=127.21.41.212:39931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:36317
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40851/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/testchainca.cert with env {}
W20260809 00:33:03.588485 26904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:03.588701 26904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:03.588723 26904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:03.589927 26904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:03.589973 26904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:03.589986 26904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:03.589999 26904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:03.590011 26904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:33:03.590024 26904 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:33:03.590035 26904 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40851/jwks.json
I20260809 00:33:03.591696 26904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:36317
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:36317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40851/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:03.591970 26904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:03.592196 26904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:03.597498 26910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:03.597509 26912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:03.597513 26909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:03.597563 26904 server_base.cc:1034] running on GCE node
I20260809 00:33:03.598027 26904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:03.598325 26904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:03.599531 26904 hybrid_clock.cc:648] HybridClock initialized: now 1786235583599473 us; error 75 us; skew 500 ppm
I20260809 00:33:03.600785 26904 webserver.cc:460] Webserver started at http://127.21.41.254:40111/ using document root <none> and password file <none>
I20260809 00:33:03.601028 26904 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:03.601099 26904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:03.601253 26904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:03.602397 26904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "c4e9d4ace67540cdafcbb330bf199a4b"
format_stamp: "Formatted at 2026-08-09 00:33:03 on dist-test-slave-zpfg"
server_key: "eb765c59e42c4b52a11964d84d0066ec"
server_key_iv: "fdbdb3b619858fde426ce7c1f96a8bda"
server_key_version: "encryptionkey@0"
I20260809 00:33:03.602782 26904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "c4e9d4ace67540cdafcbb330bf199a4b"
format_stamp: "Formatted at 2026-08-09 00:33:03 on dist-test-slave-zpfg"
server_key: "eb765c59e42c4b52a11964d84d0066ec"
server_key_iv: "fdbdb3b619858fde426ce7c1f96a8bda"
server_key_version: "encryptionkey@0"
I20260809 00:33:03.604177 26904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:33:03.605028 26918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:03.605202 26904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:03.605335 26904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "c4e9d4ace67540cdafcbb330bf199a4b"
format_stamp: "Formatted at 2026-08-09 00:33:03 on dist-test-slave-zpfg"
server_key: "eb765c59e42c4b52a11964d84d0066ec"
server_key_iv: "fdbdb3b619858fde426ce7c1f96a8bda"
server_key_version: "encryptionkey@0"
I20260809 00:33:03.605428 26904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:03.614003 26904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:03.614408 26904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:03.614569 26904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:03.618546 26904 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:36317
I20260809 00:33:03.618583 26970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:36317 every 8 connection(s)
I20260809 00:33:03.619024 26904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:03.619721 26971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:03.621865 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 26904
I20260809 00:33:03.621968 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:03.622196 21671 external_mini_cluster.cc:1428] Setting key c15c7673ce0661788b334ef2672a4cc6
I20260809 00:33:03.623143 26971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b: Bootstrap starting.
I20260809 00:33:03.623840 26971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:03.624187 26971 log.cc:824] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:03.624982 26971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b: No bootstrap required, opened a new log
I20260809 00:33:03.627400 26971 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e9d4ace67540cdafcbb330bf199a4b" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36317 } }
I20260809 00:33:03.627581 26971 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:03.627609 26971 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e9d4ace67540cdafcbb330bf199a4b, State: Initialized, Role: FOLLOWER
I20260809 00:33:03.627756 26971 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e9d4ace67540cdafcbb330bf199a4b" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36317 } }
I20260809 00:33:03.627837 26971 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:03.627859 26971 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:03.627926 26971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:03.628427 26974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.622790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39206 (local address 127.21.41.254:36317)
0809 00:33:03.622981 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:03.622989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:03.623015 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:33:03.623496 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:03.623504 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:03.623569 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:03.623687 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:03.623968 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.625085 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.626177 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.626642 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.626736 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:03.627496 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:03.627500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:03.627500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:03.627764 (+ 264us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:03.627805 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:03.627817 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:03.627899 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:03.628100 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 00:33:03.628185 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260809 00:33:03.628600 26971 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e9d4ace67540cdafcbb330bf199a4b" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36317 } }
I20260809 00:33:03.628736 26971 leader_election.cc:302] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4e9d4ace67540cdafcbb330bf199a4b; no voters:
I20260809 00:33:03.628895 26971 leader_election.cc:288] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:03.628959 26976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:03.629151 26971 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:03.629150 26976 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [term 1 LEADER]: Becoming Leader. State: Replica: c4e9d4ace67540cdafcbb330bf199a4b, State: Running, Role: LEADER
I20260809 00:33:03.629300 26976 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e9d4ace67540cdafcbb330bf199a4b" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36317 } }
I20260809 00:33:03.629496 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:33:03.629979 26978 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e9d4ace67540cdafcbb330bf199a4b. Latest consensus state: current_term: 1 leader_uuid: "c4e9d4ace67540cdafcbb330bf199a4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e9d4ace67540cdafcbb330bf199a4b" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36317 } } }
I20260809 00:33:03.630172 26978 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:03.630832 26977 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4e9d4ace67540cdafcbb330bf199a4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e9d4ace67540cdafcbb330bf199a4b" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36317 } } }
I20260809 00:33:03.630919 26977 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b [sys.catalog]: This master's current role is: LEADER
W20260809 00:33:03.631109 26991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:33:03.631173 26991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:33:03.631323 26988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:03.631515 26988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:03.633118 26988 catalog_manager.cc:1347] Generated new cluster ID: 3ae81e721dbd4326a4455e3bcc05bdfe
I20260809 00:33:03.633181 26988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:33:03.653514 26988 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:03.654428 26988 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:03.659777 26988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4e9d4ace67540cdafcbb330bf199a4b: Generated new TSK 0
I20260809 00:33:03.660008 26988 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 00:33:03.687041 27001 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 00:33:03.687142 27001 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 00:33:03.687493 26974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.672702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39212 (local address 127.21.41.254:36317)
0809 00:33:03.672773 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:03.672775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:03.672918 (+ 143us) server_negotiation.cc:407] Connection header received
0809 00:33:03.672981 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:03.672982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:03.672996 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:03.673082 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:33:03.673219 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.674024 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.674872 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.675190 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.675265 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:03.675488 (+ 223us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:33:03.687250 (+ 11762us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40851/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 00:33:03.687415 (+ 165us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39212: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40851/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 00:33:03.687558 26974 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39212: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40851/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 00:33:03.687585 27000 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:03.672667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:36317 (local address 127.0.0.1:39212)
0809 00:33:03.672840 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:03.672852 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:03.672916 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:03.673086 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:03.673087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:03.673093 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:33:03.673153 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:03.673155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.674133 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.674134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:03.674795 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:03.674798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.675313 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.675314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:03.675368 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:03.675373 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:33:03.687376 (+ 12003us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40851/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 00:33:03.687485 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.41.254:36317: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40851/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":120,"threads_started":1}
I20260809 00:33:03.687988 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 26904
2026-08-09T00:33:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (229 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 00:33:03.726449 21671 mini_oidc.cc:152] Starting JWKS server
I20260809 00:33:03.727133 21671 webserver.cc:460] Webserver started at https://127.0.0.1:44825/ using document root <none> and password file <none>
I20260809 00:33:03.727183 21671 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:33:03.727324 21671 webserver.cc:460] Webserver started at http://127.0.0.1:45539/ using document root <none> and password file <none>
2026-08-09T00:33:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:03Z Disabled control of system clock
I20260809 00:33:03.735462 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35331
--webserver_interface=127.21.41.254
--webserver_port=0
--builtin_ntp_servers=127.21.41.212:33431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:35331
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44825/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 00:33:03.816788 27007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:03.817039 27007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:03.817075 27007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:03.818257 27007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:03.818323 27007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:03.818377 27007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:03.818408 27007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:03.818437 27007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:33:03.818459 27007 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:33:03.818480 27007 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44825/jwks.json
I20260809 00:33:03.819901 27007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:35331
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44825/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:03.820201 27007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:03.820513 27007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:03.825495 27012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:03.825524 27013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:03.825510 27015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:03.825664 27007 server_base.cc:1034] running on GCE node
I20260809 00:33:03.825913 27007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:03.826236 27007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:03.827405 27007 hybrid_clock.cc:648] HybridClock initialized: now 1786235583827400 us; error 40 us; skew 500 ppm
I20260809 00:33:03.828756 27007 webserver.cc:460] Webserver started at http://127.21.41.254:46087/ using document root <none> and password file <none>
I20260809 00:33:03.829006 27007 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:03.829078 27007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:03.829253 27007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:03.830415 27007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "0f95943832e94de09bda30022d7a200d"
format_stamp: "Formatted at 2026-08-09 00:33:03 on dist-test-slave-zpfg"
server_key: "cbc9d561c59c6276e1eaa4dd77548e29"
server_key_iv: "ff3f740972c6c06e74e68b66af4300b0"
server_key_version: "encryptionkey@0"
I20260809 00:33:03.830782 27007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "0f95943832e94de09bda30022d7a200d"
format_stamp: "Formatted at 2026-08-09 00:33:03 on dist-test-slave-zpfg"
server_key: "cbc9d561c59c6276e1eaa4dd77548e29"
server_key_iv: "ff3f740972c6c06e74e68b66af4300b0"
server_key_version: "encryptionkey@0"
I20260809 00:33:03.832111 27007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:03.832908 27021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:03.833091 27007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:03.833218 27007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "0f95943832e94de09bda30022d7a200d"
format_stamp: "Formatted at 2026-08-09 00:33:03 on dist-test-slave-zpfg"
server_key: "cbc9d561c59c6276e1eaa4dd77548e29"
server_key_iv: "ff3f740972c6c06e74e68b66af4300b0"
server_key_version: "encryptionkey@0"
I20260809 00:33:03.833295 27007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:03.866515 27007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:03.866889 27007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:03.867061 27007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:03.871235 27007 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:35331
I20260809 00:33:03.871276 27073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:35331 every 8 connection(s)
I20260809 00:33:03.871649 27007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:03.872337 27074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:03.875267 27074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d: Bootstrap starting.
I20260809 00:33:03.875885 27074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:03.876183 27074 log.cc:824] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:03.876909 27074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d: No bootstrap required, opened a new log
I20260809 00:33:03.878639 27074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95943832e94de09bda30022d7a200d" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35331 } }
I20260809 00:33:03.878793 27074 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:03.878816 27074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f95943832e94de09bda30022d7a200d, State: Initialized, Role: FOLLOWER
I20260809 00:33:03.878983 27074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95943832e94de09bda30022d7a200d" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35331 } }
I20260809 00:33:03.879043 27074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:03.879099 27074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:03.879153 27074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:03.879721 27074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95943832e94de09bda30022d7a200d" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35331 } }
I20260809 00:33:03.879841 27074 leader_election.cc:302] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f95943832e94de09bda30022d7a200d; no voters:
I20260809 00:33:03.880034 27074 leader_election.cc:288] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:03.880081 27077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:03.880262 27077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [term 1 LEADER]: Becoming Leader. State: Replica: 0f95943832e94de09bda30022d7a200d, State: Running, Role: LEADER
I20260809 00:33:03.880364 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 27007
I20260809 00:33:03.880368 27077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95943832e94de09bda30022d7a200d" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35331 } }
I20260809 00:33:03.880467 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:03.880661 21671 external_mini_cluster.cc:1428] Setting key e1e3ff4befb6485ccbc08ef75d7ea403
I20260809 00:33:03.880723 27078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f95943832e94de09bda30022d7a200d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95943832e94de09bda30022d7a200d" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35331 } } }
I20260809 00:33:03.880806 27078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:03.881021 27079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f95943832e94de09bda30022d7a200d. Latest consensus state: current_term: 1 leader_uuid: "0f95943832e94de09bda30022d7a200d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95943832e94de09bda30022d7a200d" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35331 } } }
I20260809 00:33:03.881088 27079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:03.881984 27074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:03.882086 27081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:03.882769 27081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:03.884452 27081 catalog_manager.cc:1347] Generated new cluster ID: 57de2e3bf5e64c4fa17725cdc08b1e31
I20260809 00:33:03.884507 27081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:33:03.887555 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.881367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.21.41.254:35331)
0809 00:33:03.881916 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:03.881923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:03.881950 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:33:03.882067 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:03.882075 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:03.882150 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:03.882372 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:03.882712 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.884170 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.885298 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.885955 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.886084 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:03.886818 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:03.886822 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:03.886822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:03.887055 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:03.887096 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:03.887111 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:03.887183 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:03.887251 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 00:33:03.887342 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":432,"threads_started":1}
I20260809 00:33:03.888463 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:33:03.891283 27081 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:03.891944 27081 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:03.898181 27081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f95943832e94de09bda30022d7a200d: Generated new TSK 0
I20260809 00:33:03.898396 27081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:33:03.908315 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:03.904155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.21.41.254:35331)
0809 00:33:03.904220 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:03.904222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:03.904325 (+ 103us) server_negotiation.cc:407] Connection header received
0809 00:33:03.904419 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:03.904420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:03.904433 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:03.904525 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:33:03.904671 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.905165 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.906005 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.906224 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.906309 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:03.906528 (+ 219us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:33:03.908084 (+ 1556us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44825/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 00:33:03.908223 (+ 139us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49644: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44825/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 00:33:03.908380 27084 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49644: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44825/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 00:33:03.908421 27103 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:03.904089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35331 (local address 127.0.0.1:49644)
0809 00:33:03.904254 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:03.904264 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:03.904344 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:03.904527 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:03.904528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:03.904530 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:33:03.904587 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:03.904589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.905297 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.905299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:03.905915 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:03.905917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:03.906314 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:03.906316 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:03.906394 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:03.906400 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:33:03.908201 (+ 1801us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44825/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 00:33:03.908318 (+ 117us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.41.254:35331: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44825/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
I20260809 00:33:03.908958 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 27007
E20260809 00:33:03.922894 27104 webserver.cc:533] Webserver: sslize failed:
E20260809 00:33:03.922976 27104 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T00:33:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (220 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T00:33:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:03Z Disabled control of system clock
I20260809 00:33:03.954746 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:37421
--webserver_interface=127.21.41.254
--webserver_port=0
--builtin_ntp_servers=127.21.41.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:37421
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/testchainca.cert with env {}
W20260809 00:33:04.037640 27108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:04.037853 27108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:04.037875 27108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:04.039016 27108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:04.039062 27108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:04.039075 27108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:04.039088 27108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:04.039100 27108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:33:04.039113 27108 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:33:04.039124 27108 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 00:33:04.040550 27108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:37421
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:37421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:04.040756 27108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:04.040925 27108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:04.045877 27116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:04.046051 27108 server_base.cc:1034] running on GCE node
W20260809 00:33:04.045867 27114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:04.045854 27113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:04.046437 27108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:04.046689 27108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:04.047830 27108 hybrid_clock.cc:648] HybridClock initialized: now 1786235584047818 us; error 28 us; skew 500 ppm
I20260809 00:33:04.049057 27108 webserver.cc:460] Webserver started at http://127.21.41.254:46547/ using document root <none> and password file <none>
I20260809 00:33:04.049285 27108 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:04.049328 27108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:04.049417 27108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:04.050510 27108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "7c544c425b1d4d80a1f180640992f0cc"
format_stamp: "Formatted at 2026-08-09 00:33:04 on dist-test-slave-zpfg"
server_key: "cdad22afba951865ff3ace1671533655"
server_key_iv: "d7077256fb183d209e19d95550afc58f"
server_key_version: "encryptionkey@0"
I20260809 00:33:04.050828 27108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "7c544c425b1d4d80a1f180640992f0cc"
format_stamp: "Formatted at 2026-08-09 00:33:04 on dist-test-slave-zpfg"
server_key: "cdad22afba951865ff3ace1671533655"
server_key_iv: "d7077256fb183d209e19d95550afc58f"
server_key_version: "encryptionkey@0"
I20260809 00:33:04.052098 27108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:04.052759 27122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:04.052913 27108 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:04.052999 27108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "7c544c425b1d4d80a1f180640992f0cc"
format_stamp: "Formatted at 2026-08-09 00:33:04 on dist-test-slave-zpfg"
server_key: "cdad22afba951865ff3ace1671533655"
server_key_iv: "d7077256fb183d209e19d95550afc58f"
server_key_version: "encryptionkey@0"
I20260809 00:33:04.053054 27108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:04.063643 27108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:04.063921 27108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:04.064028 27108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:04.067852 27108 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:37421
I20260809 00:33:04.067930 27174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:37421 every 8 connection(s)
I20260809 00:33:04.068270 27108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:04.069106 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 27108
I20260809 00:33:04.069190 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:04.069310 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:04.069409 21671 external_mini_cluster.cc:1428] Setting key e787088590bf324fd510e43c5b791c7f
I20260809 00:33:04.072969 27175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc: Bootstrap starting.
I20260809 00:33:04.073580 27175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:04.073886 27175 log.cc:824] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:04.074702 27175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc: No bootstrap required, opened a new log
I20260809 00:33:04.075865 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:04.069957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59868 (local address 127.21.41.254:37421)
0809 00:33:04.070178 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:04.070188 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:04.070222 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:33:04.071025 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:04.071035 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:04.071119 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:04.071270 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:04.071458 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:04.072863 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:04.073771 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:04.074242 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:04.074412 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:04.075124 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:04.075127 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:04.075127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:04.075287 (+ 160us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:04.075320 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:04.075331 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:04.075386 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:04.075489 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 00:33:04.075651 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260809 00:33:04.077000 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:33:04.077184 27175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c544c425b1d4d80a1f180640992f0cc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37421 } }
I20260809 00:33:04.077340 27175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:04.077385 27175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c544c425b1d4d80a1f180640992f0cc, State: Initialized, Role: FOLLOWER
I20260809 00:33:04.077509 27175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c544c425b1d4d80a1f180640992f0cc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37421 } }
I20260809 00:33:04.077585 27175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:04.077629 27175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:04.077677 27175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:04.078429 27175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c544c425b1d4d80a1f180640992f0cc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37421 } }
I20260809 00:33:04.078562 27175 leader_election.cc:302] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c544c425b1d4d80a1f180640992f0cc; no voters:
I20260809 00:33:04.078758 27175 leader_election.cc:288] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:04.078796 27180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:04.078912 27180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [term 1 LEADER]: Becoming Leader. State: Replica: 7c544c425b1d4d80a1f180640992f0cc, State: Running, Role: LEADER
I20260809 00:33:04.079015 27175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:04.078989 27180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c544c425b1d4d80a1f180640992f0cc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37421 } }
I20260809 00:33:04.079411 27182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c544c425b1d4d80a1f180640992f0cc. Latest consensus state: current_term: 1 leader_uuid: "7c544c425b1d4d80a1f180640992f0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c544c425b1d4d80a1f180640992f0cc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37421 } } }
I20260809 00:33:04.079421 27181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c544c425b1d4d80a1f180640992f0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c544c425b1d4d80a1f180640992f0cc" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37421 } } }
I20260809 00:33:04.079576 27181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:04.079592 27182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:04.079977 27187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:04.080488 27187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:04.082058 27187 catalog_manager.cc:1347] Generated new cluster ID: 387d94ca38824455a3b655d830383521
I20260809 00:33:04.082113 27187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:33:04.119683 27187 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:04.120421 27187 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:04.136451 27187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c544c425b1d4d80a1f180640992f0cc: Generated new TSK 0
I20260809 00:33:04.136691 27187 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:33:04.615212 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:04.147634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59880 (local address 127.21.41.254:37421)
0809 00:33:04.147699 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:04.147701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:04.147939 (+ 238us) server_negotiation.cc:407] Connection header received
0809 00:33:04.148041 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:04.148042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:04.148056 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:04.148168 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:33:04.148316 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:04.148857 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:04.149692 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:04.149891 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:04.149969 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:04.150177 (+ 208us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:33:04.614970 (+464793us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 00:33:04.615148 (+ 178us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59880: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 00:33:04.615293 27178 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59880: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 00:33:04.615387 27204 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:04.147577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:37421 (local address 127.0.0.1:59880)
0809 00:33:04.147844 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:04.147856 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:04.147959 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:04.148168 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:04.148169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:04.148171 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:33:04.148234 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:04.148236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:04.148962 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:04.148963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:04.149600 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:04.149602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:04.149978 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:04.149979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:04.150047 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:04.150057 (+ 10us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:33:04.615121 (+465064us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 00:33:04.615250 (+ 129us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.41.254:37421: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":102,"threads_started":1}
I20260809 00:33:04.615984 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 27108
2026-08-09T00:33:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (687 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (94 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:33:04 dist-test-slave-zpfg krb5kdc[27216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:33:04 dist-test-slave-zpfg krb5kdc[27216](info): set up 2 sockets
Aug 09 00:33:04 dist-test-slave-zpfg krb5kdc[27216](info): commencing operation
krb5kdc: starting...
W20260809 00:33:05.836369 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:33:05 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:33:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:33:05.961364 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35033
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:35033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:06.045720 27229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:06.046027 27229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:06.046066 27229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:06.047371 27229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:06.047420 27229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:06.047435 27229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:06.047448 27229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:06.047461 27229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:06.048842 27229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:35033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:06.049027 27229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:06.049192 27229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:06.054191 27235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:06.054191 27237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:06.054219 27234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:06.054383 27229 server_base.cc:1034] running on GCE node
I20260809 00:33:06.054729 27229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:06.055060 27229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:06.056321 27229 hybrid_clock.cc:648] HybridClock initialized: now 1786235586056321 us; error 44 us; skew 500 ppm
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:06.059237 27229 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:33:06.059759 27229 webserver.cc:460] Webserver started at http://127.21.41.254:35161/ using document root <none> and password file <none>
I20260809 00:33:06.059998 27229 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:06.060065 27229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:06.060220 27229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:06.061323 27229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "32fc5b2169b54e0d8dbadc8287f0adcf"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "a3d1c3c6083a2ab5142f88aa82e2cf22"
server_key_iv: "65aff05befb0b4b221e843b534d7c0ec"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.061710 27229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "32fc5b2169b54e0d8dbadc8287f0adcf"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "a3d1c3c6083a2ab5142f88aa82e2cf22"
server_key_iv: "65aff05befb0b4b221e843b534d7c0ec"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.063247 27229 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:33:06.064024 27244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:06.064195 27229 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:06.064322 27229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "32fc5b2169b54e0d8dbadc8287f0adcf"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "a3d1c3c6083a2ab5142f88aa82e2cf22"
server_key_iv: "65aff05befb0b4b221e843b534d7c0ec"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.064399 27229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:06.092046 27229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:06.094475 27229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:06.094640 27229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:06.098677 27229 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:35033
I20260809 00:33:06.098731 27296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:35033 every 8 connection(s)
I20260809 00:33:06.099113 27229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:06.099988 27297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:06.102841 27297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf: Bootstrap starting.
I20260809 00:33:06.103582 27297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:06.103884 27297 log.cc:824] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:06.104789 27297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf: No bootstrap required, opened a new log
I20260809 00:33:06.106390 27297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32fc5b2169b54e0d8dbadc8287f0adcf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35033 } }
I20260809 00:33:06.106602 27297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:06.106647 27297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fc5b2169b54e0d8dbadc8287f0adcf, State: Initialized, Role: FOLLOWER
I20260809 00:33:06.106664 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 27229
I20260809 00:33:06.106772 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:06.106817 27297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32fc5b2169b54e0d8dbadc8287f0adcf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35033 } }
I20260809 00:33:06.106904 27297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:06.106941 27297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:06.106974 21671 external_mini_cluster.cc:1428] Setting key 89fbe9ec2210009f3e05a280a8c8e508
I20260809 00:33:06.106992 27297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:06.107543 27297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32fc5b2169b54e0d8dbadc8287f0adcf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35033 } }
I20260809 00:33:06.107666 27297 leader_election.cc:302] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32fc5b2169b54e0d8dbadc8287f0adcf; no voters:
I20260809 00:33:06.107827 27297 leader_election.cc:288] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:06.108052 27302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:06.108112 27297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:06.108167 27302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [term 1 LEADER]: Becoming Leader. State: Replica: 32fc5b2169b54e0d8dbadc8287f0adcf, State: Running, Role: LEADER
I20260809 00:33:06.108318 27302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32fc5b2169b54e0d8dbadc8287f0adcf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35033 } }
I20260809 00:33:06.108701 27302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32fc5b2169b54e0d8dbadc8287f0adcf. Latest consensus state: current_term: 1 leader_uuid: "32fc5b2169b54e0d8dbadc8287f0adcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32fc5b2169b54e0d8dbadc8287f0adcf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35033 } } }
I20260809 00:33:06.108965 27302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:06.108877 27303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32fc5b2169b54e0d8dbadc8287f0adcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32fc5b2169b54e0d8dbadc8287f0adcf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35033 } } }
I20260809 00:33:06.109385 27303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:06.109573 27311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:06.110071 27311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:06.111588 27311 catalog_manager.cc:1347] Generated new cluster ID: a333ea345dc540b1ac5fb95a0c94c480
I20260809 00:33:06.111640 27311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:06.119163 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:06.107597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56224 (local address 127.21.41.254:35033)
0809 00:33:06.107774 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:06.107783 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:06.107819 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:33:06.107941 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:06.107948 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:06.108009 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:06.108940 (+ 931us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:06.109053 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.110866 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.111889 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.112484 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.112583 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:06.115474 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:06.115478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:06.115478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:06.115490 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:06.117406 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.117825 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.117827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.117828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.117934 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.118178 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.118179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.118180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.118539 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:06.118733 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:06.118809 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 00:33:06.118954 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260809 00:33:06.132561 27311 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:06.133340 27311 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:06.142973 27311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32fc5b2169b54e0d8dbadc8287f0adcf: Generated new TSK 0
I20260809 00:33:06.143187 27311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:33:06.173102 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35033
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:06.255633 27323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:06.255869 27323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:06.255906 27323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:06.257050 27323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:06.257114 27323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:06.257157 27323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:33:06.258684 27323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:06.258961 27323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:06.259218 27323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:06.264904 27331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:06.264904 27329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:06.264896 27328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:06.265251 27323 server_base.cc:1034] running on GCE node
I20260809 00:33:06.265465 27323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:06.265708 27323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:06.266856 27323 hybrid_clock.cc:648] HybridClock initialized: now 1786235586266842 us; error 26 us; skew 500 ppm
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:06.269563 27323 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:33:06.270076 27323 webserver.cc:460] Webserver started at http://127.21.41.193:42837/ using document root <none> and password file <none>
I20260809 00:33:06.270313 27323 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:06.270421 27323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:06.270586 27323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:06.271673 27323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "797d0c241047465686734eefdefa9ac3"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "abb4771eec582a59ba3832249ce58ecd"
server_key_iv: "9b506c6f70f3a3ce9ec35a3a1980aad1"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.272085 27323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "797d0c241047465686734eefdefa9ac3"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "abb4771eec582a59ba3832249ce58ecd"
server_key_iv: "9b506c6f70f3a3ce9ec35a3a1980aad1"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.273612 27323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:33:06.274328 27338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:06.274507 27323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:06.274603 27323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "797d0c241047465686734eefdefa9ac3"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "abb4771eec582a59ba3832249ce58ecd"
server_key_iv: "9b506c6f70f3a3ce9ec35a3a1980aad1"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.274688 27323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:06.291054 27323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:06.292908 27323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:06.293077 27323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:06.293331 27323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:06.293642 27323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:06.293695 27323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:06.293735 27323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:06.293764 27323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:06.299899 27323 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:35359
I20260809 00:33:06.299954 27452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:35359 every 8 connection(s)
I20260809 00:33:06.300307 27323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:06.308815 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 27323
I20260809 00:33:06.308929 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:33:06.309172 21671 external_mini_cluster.cc:1428] Setting key 819e5d34c67200739012180eb6cfa4e7
I20260809 00:33:06.310068 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:06.301093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:33213 (local address 127.21.41.254:35033)
0809 00:33:06.301137 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:06.301143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:06.301374 (+ 231us) server_negotiation.cc:407] Connection header received
0809 00:33:06.302401 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:06.302402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:06.302418 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:06.302493 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:06.303155 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.303682 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.304685 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.304887 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.304972 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:06.307164 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:06.307165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:06.307166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:06.307173 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:06.308479 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.309010 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.309011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.309011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.309089 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.309394 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.309395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.309396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.309565 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:06.309638 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:06.309890 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 00:33:06.309976 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:33:06.310276 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:06.301064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35033 (local address 127.21.41.193:33213)
0809 00:33:06.301256 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:06.301291 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:06.302247 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:06.302572 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:06.302579 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:06.302846 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:06.303043 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:06.303047 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.303784 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.303785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:06.304593 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:06.304595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.304983 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.304984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:06.305112 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:06.305649 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:06.305658 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:06.307066 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:06.308576 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:06.308584 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:06.308586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:06.308908 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:06.309171 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:06.309171 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:06.309172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:06.309315 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:06.309629 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:06.309635 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:06.309752 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 00:33:06.309866 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 00:33:06.310024 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
I20260809 00:33:06.310870 27453 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35033
I20260809 00:33:06.310976 27453 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:06.311151 27453 heartbeater.cc:511] Master 127.21.41.254:35033 requested a full tablet report, sending...
I20260809 00:33:06.311720 27261 ts_manager.cc:194] Registered new tserver with Master: 797d0c241047465686734eefdefa9ac3 (127.21.41.193:35359)
I20260809 00:33:06.312355 27261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:33213
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:33:06.336803 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35033
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:06.418713 27458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:06.418936 27458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:06.418972 27458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:06.420174 27458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:06.420229 27458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:06.420280 27458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:33:06.421818 27458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:06.422084 27458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:06.422328 27458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:06.427260 27466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:06.427260 27464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:06.427258 27463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:06.427493 27458 server_base.cc:1034] running on GCE node
I20260809 00:33:06.427718 27458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:06.427965 27458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:06.429131 27458 hybrid_clock.cc:648] HybridClock initialized: now 1786235586429130 us; error 38 us; skew 500 ppm
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:06.431653 27458 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:33:06.432070 27458 webserver.cc:460] Webserver started at http://127.21.41.194:33549/ using document root <none> and password file <none>
I20260809 00:33:06.432318 27458 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:06.432386 27458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:06.432538 27458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:06.433578 27458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "c17fe484f416443fbf80ea0bc715376f"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "be37e473ada0bfda3bc404868a61370f"
server_key_iv: "52624745d33243a53623f557ef8c64ce"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.433928 27458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "c17fe484f416443fbf80ea0bc715376f"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "be37e473ada0bfda3bc404868a61370f"
server_key_iv: "52624745d33243a53623f557ef8c64ce"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.435256 27458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:33:06.436007 27473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:06.436240 27458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:06.436347 27458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "c17fe484f416443fbf80ea0bc715376f"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "be37e473ada0bfda3bc404868a61370f"
server_key_iv: "52624745d33243a53623f557ef8c64ce"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.436420 27458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:06.451489 27458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:06.453338 27458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:06.453501 27458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:06.453737 27458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:06.454043 27458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:06.454094 27458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:06.454135 27458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:06.454162 27458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:06.459882 27458 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:36497
I20260809 00:33:06.459933 27588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:36497 every 8 connection(s)
I20260809 00:33:06.460234 27458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:33:06.461761 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 27458
I20260809 00:33:06.461864 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:33:06.462091 21671 external_mini_cluster.cc:1428] Setting key 941dce59878a95f011ee2eaca04b1d25
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
I20260809 00:33:06.470835 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:06.461057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:57901 (local address 127.21.41.254:35033)
0809 00:33:06.461122 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:06.461124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:06.461357 (+ 233us) server_negotiation.cc:407] Connection header received
0809 00:33:06.462702 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:06.462703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:06.462723 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:06.462778 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:06.463648 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.464139 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.465268 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.465458 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.465557 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:06.467943 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:06.467945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:06.467945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:06.467953 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:06.469371 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.469793 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.469794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.469794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.469852 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.470111 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.470112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.470113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.470235 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:06.470314 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:06.470645 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 00:33:06.470741 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:33:06.471022 27591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:06.461056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35033 (local address 127.21.41.194:57901)
0809 00:33:06.461239 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:06.461274 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:06.462598 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:06.462852 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:06.462855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:06.463259 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:06.463549 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:06.463554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.464235 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.464236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:06.465154 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:06.465156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.465565 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.465567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:06.465712 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:06.466237 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:06.466250 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:06.467815 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:06.469456 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:06.469464 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:06.469466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:06.469698 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:06.469916 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:06.469917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:06.469918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:06.470038 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:06.470322 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:06.470329 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:06.470506 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 00:33:06.470613 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 00:33:06.470777 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
I20260809 00:33:06.471544 27589 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35033
I20260809 00:33:06.471645 27589 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:06.471832 27589 heartbeater.cc:511] Master 127.21.41.254:35033 requested a full tablet report, sending...
I20260809 00:33:06.472232 27261 ts_manager.cc:194] Registered new tserver with Master: c17fe484f416443fbf80ea0bc715376f (127.21.41.194:36497)
I20260809 00:33:06.472596 27261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:57901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:33:06.489711 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35033
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:06.571596 27594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:06.571837 27594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:06.571875 27594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:06.573045 27594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:06.573112 27594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:06.573154 27594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:06.574677 27594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:06.574939 27594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:06.575173 27594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:06.580207 27600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:06.580221 27599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:06.580353 27602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:06.580435 27594 server_base.cc:1034] running on GCE node
I20260809 00:33:06.580719 27594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:06.580983 27594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:06.582186 27594 hybrid_clock.cc:648] HybridClock initialized: now 1786235586582173 us; error 39 us; skew 500 ppm
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:06.584861 27594 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:33:06.585347 27594 webserver.cc:460] Webserver started at http://127.21.41.195:37637/ using document root <none> and password file <none>
I20260809 00:33:06.585594 27594 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:06.585661 27594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:06.585805 27594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:06.586930 27594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "e6c6b34cc23144f9bc2476e5b5d1d310"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "db71ea862304db1490b0be769a44f9c1"
server_key_iv: "8361a26cd1bbd488ac1b4893833c2992"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.587289 27594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "e6c6b34cc23144f9bc2476e5b5d1d310"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "db71ea862304db1490b0be769a44f9c1"
server_key_iv: "8361a26cd1bbd488ac1b4893833c2992"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.588594 27594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:33:06.589324 27609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:06.589473 27594 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:06.589574 27594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "e6c6b34cc23144f9bc2476e5b5d1d310"
format_stamp: "Formatted at 2026-08-09 00:33:06 on dist-test-slave-zpfg"
server_key: "db71ea862304db1490b0be769a44f9c1"
server_key_iv: "8361a26cd1bbd488ac1b4893833c2992"
server_key_version: "encryptionkey@0"
I20260809 00:33:06.589648 27594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:06.602231 27594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:06.604102 27594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:06.604274 27594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:06.604542 27594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:06.604892 27594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:06.604945 27594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:06.604987 27594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:06.605014 27594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:06.611130 27594 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:44833
I20260809 00:33:06.611174 27723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:44833 every 8 connection(s)
I20260809 00:33:06.611490 27594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:33:06.614751 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 27594
I20260809 00:33:06.614863 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:33:06.615067 21671 external_mini_cluster.cc:1428] Setting key f15bc0ac092ef13eba9a945cb06ed3eb
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:06.621115 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:06.612152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:39763 (local address 127.21.41.254:35033)
0809 00:33:06.612193 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:06.612195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:06.612450 (+ 255us) server_negotiation.cc:407] Connection header received
0809 00:33:06.613398 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:06.613399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:06.613412 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:06.613471 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:06.614090 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.614772 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.616186 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.616391 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.616447 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:06.618563 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:06.618565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:06.618565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:06.618572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:06.620009 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.620370 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.620371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.620372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.620425 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.620625 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.620626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.620626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.620727 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:06.620780 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:06.620995 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 00:33:06.621052 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:33:06.621300 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:06.612150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35033 (local address 127.21.41.195:39763)
0809 00:33:06.612327 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:06.612362 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:06.613286 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:06.613539 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:06.613543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:06.613809 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:06.613995 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:06.613999 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.614909 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.614910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:06.616121 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:06.616124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.616460 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.616460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:06.616556 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:06.617075 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:06.617083 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:06.618483 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:06.620080 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:06.620086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:06.620088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:06.620306 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:06.620482 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:06.620482 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:06.620482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:06.620575 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:06.620779 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:06.620783 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:06.620893 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 00:33:06.620976 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:33:06.621088 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260809 00:33:06.621815 27724 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35033
I20260809 00:33:06.621922 27724 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:06.622105 27724 heartbeater.cc:511] Master 127.21.41.254:35033 requested a full tablet report, sending...
I20260809 00:33:06.622501 27261 ts_manager.cc:194] Registered new tserver with Master: e6c6b34cc23144f9bc2476e5b5d1d310 (127.21.41.195:44833)
I20260809 00:33:06.622874 27261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:39763
I20260809 00:33:06.626869 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/test-admin.keytab.
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:06.642838 21671 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 00:33:06.644457 27731 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:06.643724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47106 (local address 127.21.41.193:35359)
0809 00:33:06.643893 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:06.643897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:06.643907 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:06.643932 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:06.643934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:06.643948 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:06.644022 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:06.644350 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47106: BlockingRecv error: recv got EOF from 127.0.0.1:47106 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
W20260809 00:33:06.644551 21671 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5cc)
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:06.645476 21671 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:06.646430 21671 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.193@KRBTEST.COM
I20260809 00:33:06.655385 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:06.647024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.21.41.193:35359)
0809 00:33:06.647104 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:06.647107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:06.647121 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:33:06.647239 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:06.647240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:06.647252 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:06.647328 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:06.647895 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.648705 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.649552 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.649915 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.650003 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:06.652277 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:06.652287 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:06.652287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:06.652296 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:06.654009 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.654457 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.654461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.654461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.654528 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.654750 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.654751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.654751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.654904 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:06.655087 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:06.655166 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 00:33:06.655254 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 00:33:06.657373 27731 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:06.656691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47126 (local address 127.21.41.193:35359)
0809 00:33:06.656730 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:06.656732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:06.656761 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:33:06.656820 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:06.656821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:06.656829 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:06.656876 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:06.657303 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47126: BlockingRecv error: recv got EOF from 127.0.0.1:47126 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:33:06.657420 21671 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5cc' of type FILE
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:06.658479 21671 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:06.659384 21671 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.193@KRBTEST.COM
I20260809 00:33:06.667272 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:06.659942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.21.41.193:35359)
0809 00:33:06.660103 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:06.660105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:06.660184 (+ 79us) server_negotiation.cc:407] Connection header received
0809 00:33:06.660289 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:06.660290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:06.660302 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:06.660376 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:06.660864 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.661397 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.662152 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.662469 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.662561 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:06.664626 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:06.664627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:06.664627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:06.664635 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:06.665891 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.666299 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.666300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.666301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.666414 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.666658 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.666659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.666659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.666777 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:06.666923 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:06.667030 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 00:33:06.667137 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
W20260809 00:33:06.668845 27731 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:06.668334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47150 (local address 127.21.41.193:35359)
0809 00:33:06.668396 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:06.668398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:06.668426 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:33:06.668494 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:06.668494 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:06.668502 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:06.668547 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:06.668794 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47150: BlockingRecv error: recv got EOF from 127.0.0.1:47150 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 00:33:06.668942 21671 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786235559853195-21671-0/krb5kdc/krb5cc' of type FILE
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:06.670046 21671 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:06.671047 21671 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.193@KRBTEST.COM
I20260809 00:33:06.677860 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:06.671750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47156 (local address 127.21.41.193:35359)
0809 00:33:06.671806 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:06.671813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:06.671821 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:33:06.671837 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:06.671838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:06.671845 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:06.671893 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:06.672239 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.672701 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.673363 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:06.673540 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:06.673597 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:06.675523 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:06.675524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:06.675524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:06.675530 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:06.676996 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.677297 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.677298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.677299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.677348 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:06.677492 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:06.677492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:06.677493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:06.677571 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:06.677672 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:06.677712 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 00:33:06.677765 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:33:06.678584 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 27323
I20260809 00:33:06.683775 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 27458
I20260809 00:33:06.688174 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 27594
I20260809 00:33:06.692546 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 27229
2026-08-09T00:33:06Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1891 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27744](info): set up 2 sockets
Aug 09 00:33:06 dist-test-slave-zpfg krb5kdc[27744](info): commencing operation
krb5kdc: starting...
W20260809 00:33:07.748114 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:33:07 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:33:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.254.keytab.
Entry for principal oryx/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.254.keytab.
I20260809 00:33:07.871083 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:46427
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.254.keytab
--principal=oryx/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:46427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:07.954612 27757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:07.954828 27757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:07.954850 27757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:07.956105 27757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:07.956146 27757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:07.956161 27757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:07.956176 27757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:07.956224 27757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:07.957691 27757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:46427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.254.keytab
--principal=oryx/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:46427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:07.957940 27757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:07.958177 27757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:07.963191 27763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:07.963389 27757 server_base.cc:1034] running on GCE node
W20260809 00:33:07.963199 27765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:07.963248 27762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:07.963742 27757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:07.964010 27757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:07.965152 27757 hybrid_clock.cc:648] HybridClock initialized: now 1786235587965129 us; error 39 us; skew 500 ppm
Aug 09 00:33:07 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235587, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:07.972957 27757 init.cc:377] Logged in from keytab as oryx/127.21.41.254@KRBTEST.COM (short username oryx)
I20260809 00:33:07.973538 27757 webserver.cc:460] Webserver started at http://127.21.41.254:44611/ using document root <none> and password file <none>
I20260809 00:33:07.973774 27757 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:07.973819 27757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:07.973995 27757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:07.975214 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "d4914f52a2dc4d7eaf6c0d04db25f582"
format_stamp: "Formatted at 2026-08-09 00:33:07 on dist-test-slave-zpfg"
server_key: "f9f77d670caf5acf54e572b1e3b88045"
server_key_iv: "d8e6777c9ecf3b47aae6204405c7f3fc"
server_key_version: "encryptionkey@0"
I20260809 00:33:07.975606 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "d4914f52a2dc4d7eaf6c0d04db25f582"
format_stamp: "Formatted at 2026-08-09 00:33:07 on dist-test-slave-zpfg"
server_key: "f9f77d670caf5acf54e572b1e3b88045"
server_key_iv: "d8e6777c9ecf3b47aae6204405c7f3fc"
server_key_version: "encryptionkey@0"
I20260809 00:33:07.977005 27757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:33:07.977766 27772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:07.977946 27757 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:07.978056 27757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "d4914f52a2dc4d7eaf6c0d04db25f582"
format_stamp: "Formatted at 2026-08-09 00:33:07 on dist-test-slave-zpfg"
server_key: "f9f77d670caf5acf54e572b1e3b88045"
server_key_iv: "d8e6777c9ecf3b47aae6204405c7f3fc"
server_key_version: "encryptionkey@0"
I20260809 00:33:07.978133 27757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:07.988919 27757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:07.991447 27757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:07.991600 27757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:07.996078 27757 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:46427
I20260809 00:33:07.996167 27824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:46427 every 8 connection(s)
I20260809 00:33:07.996479 27757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:07.997473 27825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:07.999898 27825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582: Bootstrap starting.
I20260809 00:33:08.000530 27825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:08.000825 27825 log.cc:824] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:08.001549 27825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582: No bootstrap required, opened a new log
I20260809 00:33:08.003253 27825 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } }
I20260809 00:33:08.003392 27825 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:08.003414 27825 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4914f52a2dc4d7eaf6c0d04db25f582, State: Initialized, Role: FOLLOWER
I20260809 00:33:08.003589 27825 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } }
I20260809 00:33:08.003649 27825 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:08.003670 27825 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:08.003726 27825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:08.004557 27825 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } }
I20260809 00:33:08.004647 27825 leader_election.cc:302] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4914f52a2dc4d7eaf6c0d04db25f582; no voters:
I20260809 00:33:08.004804 27825 leader_election.cc:288] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:08.004868 27828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:08.005077 27828 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 1 LEADER]: Becoming Leader. State: Replica: d4914f52a2dc4d7eaf6c0d04db25f582, State: Running, Role: LEADER
I20260809 00:33:08.005119 27825 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:08.005180 27828 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } }
I20260809 00:33:08.005769 27830 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4914f52a2dc4d7eaf6c0d04db25f582. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } }
I20260809 00:33:08.005883 27830 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:08.005782 27829 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } }
I20260809 00:33:08.005955 27829 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:08.006484 27837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:08.006639 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 27757
I20260809 00:33:08.006767 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:08.006953 21671 external_mini_cluster.cc:1428] Setting key d3dd574d268570e57ecf589bc992aa6f
I20260809 00:33:08.007180 27837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:08.008724 27837 catalog_manager.cc:1347] Generated new cluster ID: 5495401561c54a1887cfc553204f7b88
I20260809 00:33:08.008795 27837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:33:08.015653 27837 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:08.016341 27837 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:08.022789 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.007468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.21.41.254:46427)
0809 00:33:08.007639 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.007648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.012889 (+ 5241us) server_negotiation.cc:407] Connection header received
0809 00:33:08.013024 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.013032 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.013091 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.013241 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.014083 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.015525 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.016437 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.016867 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.017004 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.019385 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.019389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.019389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.019402 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.021142 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.021503 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.021506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.021507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.021601 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.021843 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.021844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.021844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.022074 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.022378 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.022454 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.022596 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260809 00:33:08.026178 27837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582: Generated new TSK 0
I20260809 00:33:08.026376 27837 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab.
Entry for principal oryx/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab.
I20260809 00:33:08.055256 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46427
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab
--principal=oryx/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:08.140865 27851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:08.141104 27851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:08.141146 27851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:08.142295 27851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:08.142379 27851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:08.142426 27851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:33:08.143882 27851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab
--principal=oryx/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:08.144136 27851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:08.144389 27851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:08.149565 27856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:08.149709 27859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:08.149574 27857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:08.149849 27851 server_base.cc:1034] running on GCE node
I20260809 00:33:08.150071 27851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:08.150382 27851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:08.151544 27851 hybrid_clock.cc:648] HybridClock initialized: now 1786235588151537 us; error 35 us; skew 500 ppm
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:08.154027 27851 init.cc:377] Logged in from keytab as oryx/127.21.41.193@KRBTEST.COM (short username oryx)
I20260809 00:33:08.154579 27851 webserver.cc:460] Webserver started at http://127.21.41.193:44957/ using document root <none> and password file <none>
I20260809 00:33:08.154809 27851 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:08.154879 27851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:08.155040 27851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:08.156129 27851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "660fd2a7b01b4e15a201d53f7fe61e58"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "8109878b4a47237e01dfffe6f36b971a"
server_key_iv: "95b8f721cd766c33bb7833fd43b4942c"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.156510 27851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "660fd2a7b01b4e15a201d53f7fe61e58"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "8109878b4a47237e01dfffe6f36b971a"
server_key_iv: "95b8f721cd766c33bb7833fd43b4942c"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.157832 27851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:33:08.158635 27866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.158810 27851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:08.158910 27851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "660fd2a7b01b4e15a201d53f7fe61e58"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "8109878b4a47237e01dfffe6f36b971a"
server_key_iv: "95b8f721cd766c33bb7833fd43b4942c"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.158994 27851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:08.176172 27851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:08.178236 27851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:08.178460 27851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:08.178707 27851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:08.179029 27851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:08.179086 27851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.179131 27851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:08.179155 27851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.186017 27851 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:38675
I20260809 00:33:08.186074 27979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:38675 every 8 connection(s)
I20260809 00:33:08.186499 27851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:08.193341 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 27851
I20260809 00:33:08.193461 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:33:08.193679 21671 external_mini_cluster.cc:1428] Setting key ab23ada1606d09542bf5d5ccd941bd30
I20260809 00:33:08.196172 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.187279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:49801 (local address 127.21.41.254:46427)
0809 00:33:08.187325 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.187327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.187506 (+ 179us) server_negotiation.cc:407] Connection header received
0809 00:33:08.188005 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.188006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.188015 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.188070 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.188970 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.189474 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.190598 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.190806 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.190855 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.193049 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.193050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.193051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.193057 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.194728 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.195156 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.195157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.195157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.195214 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.195491 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.195492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.195493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.195622 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.195703 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.195982 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.196081 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:33:08.196343 27982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.187233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.193:49801)
0809 00:33:08.187394 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.187436 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.187923 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.188130 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.188134 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.188701 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.188893 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.188898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.189638 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.189638 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.190509 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.190512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.190875 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.190876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.190969 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.191427 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.191435 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.192957 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.194834 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.194841 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.194843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.195069 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.195316 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.195317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.195317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.195424 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.195692 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.195699 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.195840 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.195943 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.196100 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260809 00:33:08.196779 27980 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46427
I20260809 00:33:08.196887 27980 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:08.197077 27980 heartbeater.cc:511] Master 127.21.41.254:46427 requested a full tablet report, sending...
I20260809 00:33:08.197633 27789 ts_manager.cc:194] Registered new tserver with Master: 660fd2a7b01b4e15a201d53f7fe61e58 (127.21.41.193:38675)
I20260809 00:33:08.198324 27789 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:49801
WARNING: no policy specified for oryx/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab.
Entry for principal oryx/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab.
I20260809 00:33:08.221643 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46427
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab
--principal=oryx/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:08.305833 27985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:08.306094 27985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:08.306118 27985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:08.307368 27985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:08.307410 27985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:08.307436 27985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:33:08.308835 27985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab
--principal=oryx/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:08.309026 27985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:08.309192 27985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:08.314484 27993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:08.314495 27991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:08.314531 27990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:08.314721 27985 server_base.cc:1034] running on GCE node
I20260809 00:33:08.315033 27985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:08.315279 27985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:08.316418 27985 hybrid_clock.cc:648] HybridClock initialized: now 1786235588316391 us; error 44 us; skew 500 ppm
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:08.318958 27985 init.cc:377] Logged in from keytab as oryx/127.21.41.194@KRBTEST.COM (short username oryx)
I20260809 00:33:08.319410 27985 webserver.cc:460] Webserver started at http://127.21.41.194:33049/ using document root <none> and password file <none>
I20260809 00:33:08.319599 27985 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:08.319638 27985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:08.319733 27985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:08.320741 27985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "873a29f3b7e0771dbbb8f4ef40b03a9d"
server_key_iv: "92fdea870c3f253ed47cd7a2439452d7"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.321040 27985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "873a29f3b7e0771dbbb8f4ef40b03a9d"
server_key_iv: "92fdea870c3f253ed47cd7a2439452d7"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.322326 27985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:33:08.323180 28000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.323354 27985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:08.323449 27985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "873a29f3b7e0771dbbb8f4ef40b03a9d"
server_key_iv: "92fdea870c3f253ed47cd7a2439452d7"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.323509 27985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:08.345283 27985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:08.347633 27985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:08.347760 27985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:08.347968 27985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:08.348315 27985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:08.348352 27985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.348376 27985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:08.348429 27985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.354964 27985 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:32937
I20260809 00:33:08.355019 28113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:32937 every 8 connection(s)
I20260809 00:33:08.355399 27985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:33:08.357203 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 27985
I20260809 00:33:08.357326 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:33:08.357509 21671 external_mini_cluster.cc:1428] Setting key ad1003d99dca5d379192dec56a9a10b7
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:08.365933 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.356266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:34187 (local address 127.21.41.254:46427)
0809 00:33:08.356352 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.356355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.356531 (+ 176us) server_negotiation.cc:407] Connection header received
0809 00:33:08.357034 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.357035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.357046 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.357170 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.358258 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.358837 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.359939 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.360137 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.360242 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.363023 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.363024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.363024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.363031 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.364462 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.364966 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.364967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.364967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.365018 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.365296 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.365297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.365297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.365399 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.365450 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.365801 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.365864 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for oryx/127.21.41.195@KRBTEST.COM; defaulting to no policy
I20260809 00:33:08.366200 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.356225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.194:34187)
0809 00:33:08.356405 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.356444 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.356916 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.357259 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.357264 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.357955 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.358147 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.358152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.358933 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.358934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.359841 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.359844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.360250 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.360251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.360395 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.360925 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.360933 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.362927 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.364547 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.364555 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.364557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.364885 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.365086 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.365086 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.365087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.365226 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.365468 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.365474 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.365650 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.365792 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.365967 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260809 00:33:08.366729 28114 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46427
I20260809 00:33:08.366828 28114 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:08.366971 28114 heartbeater.cc:511] Master 127.21.41.254:46427 requested a full tablet report, sending...
I20260809 00:33:08.367334 27789 ts_manager.cc:194] Registered new tserver with Master: cf1f7a8afa5a42a7853b7ad1c2ee7855 (127.21.41.194:32937)
I20260809 00:33:08.367735 27789 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:34187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab.
Entry for principal oryx/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab.
I20260809 00:33:08.386108 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46427
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab
--principal=oryx/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:08.467626 28119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:08.467875 28119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:08.467905 28119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:08.469095 28119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:08.469146 28119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:08.469172 28119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:08.470799 28119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab
--principal=oryx/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:08.471068 28119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:08.471323 28119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:08.476418 28124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:08.476429 28125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:08.476418 28128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:08.476771 28119 server_base.cc:1034] running on GCE node
I20260809 00:33:08.476986 28119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:08.477229 28119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:08.478399 28119 hybrid_clock.cc:648] HybridClock initialized: now 1786235588478392 us; error 38 us; skew 500 ppm
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:08.480916 28119 init.cc:377] Logged in from keytab as oryx/127.21.41.195@KRBTEST.COM (short username oryx)
I20260809 00:33:08.481447 28119 webserver.cc:460] Webserver started at http://127.21.41.195:41753/ using document root <none> and password file <none>
I20260809 00:33:08.481691 28119 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:08.481736 28119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:08.481882 28119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:08.483023 28119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "53ea8b2562624f53a4c81f0d3ee0be50"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "928ff3cf43eb910a816208f4e1d9b6c3"
server_key_iv: "443c6127795e93a60fa51321cbad66ca"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.483397 28119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "53ea8b2562624f53a4c81f0d3ee0be50"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "928ff3cf43eb910a816208f4e1d9b6c3"
server_key_iv: "443c6127795e93a60fa51321cbad66ca"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.484725 28119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:33:08.485532 28135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.485700 28119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:08.485805 28119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "53ea8b2562624f53a4c81f0d3ee0be50"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "928ff3cf43eb910a816208f4e1d9b6c3"
server_key_iv: "443c6127795e93a60fa51321cbad66ca"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.485877 28119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:08.503502 28119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:08.505671 28119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:08.505856 28119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:08.506174 28119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:08.506568 28119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:08.506625 28119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.506660 28119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:08.506690 28119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.512869 28119 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:42291
I20260809 00:33:08.512962 28249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:42291 every 8 connection(s)
I20260809 00:33:08.513280 28119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:08.521337 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 28119
I20260809 00:33:08.521451 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:33:08.521773 21671 external_mini_cluster.cc:1428] Setting key b8a5d9e569c1bb20ab4822decbf39ce9
I20260809 00:33:08.522589 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.514059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:45685 (local address 127.21.41.254:46427)
0809 00:33:08.514104 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.514107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.514295 (+ 188us) server_negotiation.cc:407] Connection header received
0809 00:33:08.514862 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.514863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.514872 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.514932 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.515884 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.516368 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.517303 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.517480 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.517531 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.519760 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.519762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.519762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.519768 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.521009 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.521403 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.521404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.521405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.521492 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.521813 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.521814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.521814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.521981 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.522056 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.522419 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.522505 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:33:08.522830 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.514019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.195:45685)
0809 00:33:08.514176 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.514221 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.514753 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.515003 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.515007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.515596 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.515806 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.515811 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.516441 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.516442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.517230 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.517232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.517549 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.517550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.517645 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.518198 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.518214 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.519615 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.521080 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.521086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.521087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.521298 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.521589 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.521591 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.521591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.521739 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.522073 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.522079 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.522246 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.522384 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.522557 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260809 00:33:08.523379 28250 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46427
I20260809 00:33:08.523461 28250 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:08.523597 28250 heartbeater.cc:511] Master 127.21.41.254:46427 requested a full tablet report, sending...
I20260809 00:33:08.523928 27789 ts_manager.cc:194] Registered new tserver with Master: 53ea8b2562624f53a4c81f0d3ee0be50 (127.21.41.195:42291)
I20260809 00:33:08.524322 27789 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:45685
I20260809 00:33:08.533661 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.41.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.253.keytab.
Entry for principal oryx/127.21.41.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.253.keytab.
I20260809 00:33:08.562000 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.253:35627
--webserver_interface=127.21.41.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.253.keytab
--principal=oryx/127.21.41.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:46427,127.21.41.253:35627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:08.644397 28255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:08.644618 28255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:08.644641 28255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:08.645841 28255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:08.645888 28255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:08.645903 28255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:08.645951 28255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:08.645977 28255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:08.647611 28255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:46427,127.21.41.253:35627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.253.keytab
--principal=oryx/127.21.41.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.253:35627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.41.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
W20260809 00:33:08.647733 28255 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.41.254:46427,127.21.41.253:35627'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 00:33:08.647931 28255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:08.648175 28255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:08.653174 28263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:08.653169 28260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:08.653286 28261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:08.653403 28255 server_base.cc:1034] running on GCE node
I20260809 00:33:08.653633 28255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:08.653927 28255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:08.655076 28255 hybrid_clock.cc:648] HybridClock initialized: now 1786235588655064 us; error 27 us; skew 500 ppm
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:08.657641 28255 init.cc:377] Logged in from keytab as oryx/127.21.41.253@KRBTEST.COM (short username oryx)
I20260809 00:33:08.658134 28255 webserver.cc:460] Webserver started at http://127.21.41.253:33077/ using document root <none> and password file <none>
I20260809 00:33:08.658428 28255 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:08.658497 28255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:08.658648 28255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:08.659673 28255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data/instance:
uuid: "b71d8be91b3648bd82ab63e06504fe5c"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "00cce369a16e02fbd95b04d55827ed1d"
server_key_iv: "97e63d42a746868d10ab78884fe16a9e"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.660051 28255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/wal/instance:
uuid: "b71d8be91b3648bd82ab63e06504fe5c"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "00cce369a16e02fbd95b04d55827ed1d"
server_key_iv: "97e63d42a746868d10ab78884fe16a9e"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.661449 28255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:33:08.662175 28270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.662395 28255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:08.662504 28255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/wal
uuid: "b71d8be91b3648bd82ab63e06504fe5c"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "00cce369a16e02fbd95b04d55827ed1d"
server_key_iv: "97e63d42a746868d10ab78884fe16a9e"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.662595 28255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:08.671365 28255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:08.673661 28255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:08.673832 28255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:08.677906 28255 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.253:35627
I20260809 00:33:08.677937 28322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.253:35627 every 8 connection(s)
I20260809 00:33:08.678272 28255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
I20260809 00:33:08.679327 28323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:08.680866 28323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.253@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:08.687286 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 28255
I20260809 00:33:08.690457 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.681594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59316 (local address 127.21.41.254:46427)
0809 00:33:08.681668 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.681670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.681924 (+ 254us) server_negotiation.cc:407] Connection header received
0809 00:33:08.682480 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.682481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.682491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.682545 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.683125 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.683615 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.684638 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.684828 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.684880 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.687189 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.687190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.687191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.687200 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.689018 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.689398 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.689399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.689399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.689463 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.689715 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.689716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.689716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.689925 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.689996 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.690247 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.690330 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:33:08.690567 28325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.681566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59316)
0809 00:33:08.681788 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.681824 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.682326 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.682608 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.682611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.682861 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.683053 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.683057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.683718 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.683719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.684553 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.684556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.684918 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.684919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.685052 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.685520 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.685528 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.687018 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.689085 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.689091 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.689093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.689319 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.689530 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.689531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.689531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.689641 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.689995 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.690000 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.690114 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.690201 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.690321 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":124,"threads_started":1}
I20260809 00:33:08.691183 28323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.41.253@KRBTEST.COM
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.253@KRBTEST.COM for oryx/127.21.41.253@KRBTEST.COM
I20260809 00:33:08.696981 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.687829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.21.41.253:35627)
0809 00:33:08.687998 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.688002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.688065 (+ 63us) server_negotiation.cc:407] Connection header received
0809 00:33:08.688099 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.688101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.688115 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.688196 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.689018 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.689683 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.690711 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.691126 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.691199 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.693989 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.693997 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.693997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.694004 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.695791 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.696145 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.696148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.696149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.696217 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.696412 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.696413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.696413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.696560 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.696744 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.696788 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.696864 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260809 00:33:08.699072 28325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.691436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35627 (local address 127.0.0.1:34746)
0809 00:33:08.691485 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.691496 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.691536 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.692080 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.692081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.692235 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.692299 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.692301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.692931 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.692933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.693430 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.693432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.693978 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.693979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.694053 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.694466 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.694470 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.695860 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.698245 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.698246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.698246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.698474 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.698620 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.698620 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.698621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.698668 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.698851 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.698852 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.698885 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.698936 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.699006 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:33:08.699174 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.691504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.21.41.253:35627)
0809 00:33:08.691869 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.691870 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.691876 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:33:08.691892 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.691892 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.691899 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.691953 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.692405 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.692852 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.693543 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.693872 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.693944 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.695944 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.695945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.695945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.695951 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.698151 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.698524 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.698524 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.698525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.698571 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.698718 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.698718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.698719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.698800 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.698845 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.698950 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.699047 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":347,"thread_start_us":116,"threads_started":1}
I20260809 00:33:08.700373 28323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: Bootstrap starting.
I20260809 00:33:08.700928 28323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:08.701148 28323 log.cc:824] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:08.701823 28323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: No bootstrap required, opened a new log
I20260809 00:33:08.703197 28323 raft_consensus.cc:348] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } }
I20260809 00:33:08.703317 28323 raft_consensus.cc:374] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:08.703338 28323 raft_consensus.cc:729] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b71d8be91b3648bd82ab63e06504fe5c, State: Initialized, Role: FOLLOWER
I20260809 00:33:08.703455 28323 consensus_queue.cc:260] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } }
I20260809 00:33:08.703692 28330 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } } }
I20260809 00:33:08.703778 28330 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:33:08.703780 28323 sys_catalog.cc:564] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: configured and running, proceeding with master startup.
W20260809 00:33:08.705339 28341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:33:08.705395 28341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:33:08.707306 28255 master_runner.cc:428] Detected that this master b71d8be91b3648bd82ab63e06504fe5c is joining an existing cluster
I20260809 00:33:08.707389 28255 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.41.253:35627
I20260809 00:33:08.714774 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.708355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.21.41.254:46427)
0809 00:33:08.708527 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.708530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.708548 (+ 18us) server_negotiation.cc:407] Connection header received
0809 00:33:08.708574 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.708574 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.708586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.708647 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.709006 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.709607 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.710390 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.710597 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.710679 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.711682 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.711683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.711684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.711693 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.713355 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.713937 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.713939 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.713939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.713989 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.714193 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.714194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.714195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.714314 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.714417 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.714577 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.714663 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260809 00:33:08.714818 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.708327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59324)
0809 00:33:08.708459 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.708470 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.708518 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.708648 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.708648 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.708833 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.708929 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.708931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.709698 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.709699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.710279 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.710280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.710681 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.710682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.710749 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.711196 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.711200 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.711577 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.713444 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.713445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.713445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.713671 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.714064 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.714064 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.714065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.714125 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.714409 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.714410 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.714460 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.714549 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.714683 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260809 00:33:08.716074 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.708511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34758 (local address 127.21.41.253:35627)
0809 00:33:08.708550 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.708552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.708787 (+ 235us) server_negotiation.cc:407] Connection header received
0809 00:33:08.708833 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.708834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.708843 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.708905 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.709257 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.709991 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.711361 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.711669 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.711741 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.712902 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.712903 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.712903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.712912 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.714670 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.715332 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.715333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.715333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.715391 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.715587 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.715588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.715588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.715692 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.715731 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.715888 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.715971 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:33:08.716384 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.708355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35627 (local address 127.0.0.1:34758)
0809 00:33:08.708714 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.708724 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.708785 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.708933 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.708934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.709111 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.709172 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.709174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.710078 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.710078 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.711279 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.711281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.711782 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.711783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.711854 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.712385 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.712389 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.712778 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.714753 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.714753 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.714754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.715243 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.715459 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.715459 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.715460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.715511 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.715773 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.715773 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.715806 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.715868 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.716296 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":319,"thread_start_us":84,"threads_started":1}
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235587, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.254@KRBTEST.COM for oryx/127.21.41.253@KRBTEST.COM
I20260809 00:33:08.724715 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.717462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35627 (local address 127.0.0.1:34760)
0809 00:33:08.717601 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.717646 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.717722 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.717865 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.717867 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.718098 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.718144 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.718145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.719045 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.719046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.719621 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.719623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.720098 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.720099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.720190 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.720654 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.720659 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.722149 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.723722 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.723748 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.723750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.723951 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.724146 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.724147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.724147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.724198 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.724452 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.724457 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.724496 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.724558 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.724645 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260809 00:33:08.724787 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.717533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34760 (local address 127.21.41.253:35627)
0809 00:33:08.717567 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.717569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.717718 (+ 149us) server_negotiation.cc:407] Connection header received
0809 00:33:08.717785 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.717785 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.717796 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.717862 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.718212 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.718963 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.719701 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.720004 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.720076 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.722235 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.722236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.722236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.722243 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.723649 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.724022 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.724023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.724024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.724080 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.724267 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.724267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.724268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.724390 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.724457 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.724614 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.724688 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:33:08.725307 27789 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.41.253:35627: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c"
member_type: NON_VOTER
last_known_addr {
host: "127.21.41.253"
port: 35627
}
attrs {
promote: true
}
}
dest_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582"
cas_config_opid_index: -1
I20260809 00:33:08.725607 27789 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } }
I20260809 00:33:08.726099 28351 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b71d8be91b3648bd82ab63e06504fe5c (127.21.41.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } }
I20260809 00:33:08.726424 28297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:08.727181 28351 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.41.253:35627
I20260809 00:33:08.727351 28297 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Refusing update from remote peer d4914f52a2dc4d7eaf6c0d04db25f582: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 00:33:08.727473 28255 master.cc:537] Master@127.21.41.253:35627 shutting down...
I20260809 00:33:08.727450 28350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } }
I20260809 00:33:08.727653 28351 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:33:08.728396 27775 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 00:33:08.727525 28350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: This master's current role is: LEADER
W20260809 00:33:08.728955 27775 consensus_peers.cc:487] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 -> Peer b71d8be91b3648bd82ab63e06504fe5c (127.21.41.253:35627): Couldn't send request to peer b71d8be91b3648bd82ab63e06504fe5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 00:33:08.730374 28255 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:08.730463 28255 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:08.730515 28255 tablet_replica.cc:330] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: stopping tablet replica
I20260809 00:33:08.743110 28255 master.cc:552] Master@127.21.41.253:35627 shutdown complete.
I20260809 00:33:08.743187 28255 master_runner.cc:305] Clearing existing system tablet
I20260809 00:33:08.743321 28255 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:08.743386 28255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:08.744057 28255 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.002s
I20260809 00:33:08.744546 28355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.744716 28255 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:08.744796 28255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:08.744822 28255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/wal
uuid: "b71d8be91b3648bd82ab63e06504fe5c"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "00cce369a16e02fbd95b04d55827ed1d"
server_key_iv: "97e63d42a746868d10ab78884fe16a9e"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.745705 28255 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:08.747887 28255 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 00:33:08.747992 28255 log.cc:1196] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 00:33:08.748212 28255 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: Deleting consensus metadata
I20260809 00:33:08.748339 28255 master_runner.cc:315] Copying system tablet from 127.21.41.254:46427
I20260809 00:33:08.749001 28255 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: tablet copy: Beginning tablet copy session from remote peer at address 127.21.41.254:46427
I20260809 00:33:08.755574 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.749293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59338)
0809 00:33:08.749505 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.749515 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.749587 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.749725 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.749726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.749884 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.749941 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.749943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.750906 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.750907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.751474 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.751476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.751949 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.751949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.751999 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.752504 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.752509 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.752821 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.754568 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.754569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.754569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.754741 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.754944 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.754944 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.754945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.754994 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.755260 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.755260 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.755327 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.755405 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.755486 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
I20260809 00:33:08.755640 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.749427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.21.41.254:46427)
0809 00:33:08.749502 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.749504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.749583 (+ 79us) server_negotiation.cc:407] Connection header received
0809 00:33:08.749639 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.749640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.749652 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.749722 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.750007 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.750817 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.751558 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.751874 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.751945 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.752909 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.752910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.752911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.752918 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.754494 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.754817 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.754818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.754818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.754880 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.755058 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.755059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.755059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.755195 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.755266 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.755452 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.755532 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:33:08.756007 27809 tablet_copy_service.cc:140] P d4914f52a2dc4d7eaf6c0d04db25f582: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b71d8be91b3648bd82ab63e06504fe5c ({username='oryx', principal='oryx/127.21.41.253@KRBTEST.COM'} at 127.0.0.1:59338)
I20260809 00:33:08.756150 27809 tablet_copy_service.cc:161] P d4914f52a2dc4d7eaf6c0d04db25f582: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b71d8be91b3648bd82ab63e06504fe5c at {username='oryx', principal='oryx/127.21.41.253@KRBTEST.COM'} at 127.0.0.1:59338: session id = b71d8be91b3648bd82ab63e06504fe5c-00000000000000000000000000000000
I20260809 00:33:08.756946 27809 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 00:33:08.757616 28255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 00:33:08.759048 28255 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: tablet copy: Starting download of 0 data blocks...
I20260809 00:33:08.759207 28255 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: tablet copy: Starting download of 1 WAL segments...
I20260809 00:33:08.760257 28255 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 00:33:08.761104 27809 tablet_copy_service.cc:342] P d4914f52a2dc4d7eaf6c0d04db25f582: Request end of tablet copy session b71d8be91b3648bd82ab63e06504fe5c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.41.253@KRBTEST.COM'} at 127.0.0.1:59338
I20260809 00:33:08.761154 27809 tablet_copy_service.cc:433] P d4914f52a2dc4d7eaf6c0d04db25f582: ending tablet copy session b71d8be91b3648bd82ab63e06504fe5c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b71d8be91b3648bd82ab63e06504fe5c
I20260809 00:33:08.763756 28255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:08.764972 28362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:08.764984 28363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:08.765069 28365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:08.765259 28255 server_base.cc:1034] running on GCE node
I20260809 00:33:08.765342 28255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:08.765473 28255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:08.766597 28255 hybrid_clock.cc:648] HybridClock initialized: now 1786235588766590 us; error 36 us; skew 500 ppm
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:08.767931 28255 init.cc:377] Logged in from keytab as oryx/127.21.41.253@KRBTEST.COM (short username oryx)
I20260809 00:33:08.768213 28255 webserver.cc:460] Webserver started at http://127.21.41.253:44067/ using document root <none> and password file <none>
I20260809 00:33:08.768342 28255 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:08.768386 28255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:08.769013 28255 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.769550 28372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.769672 28255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.769719 28255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/wal
uuid: "b71d8be91b3648bd82ab63e06504fe5c"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "00cce369a16e02fbd95b04d55827ed1d"
server_key_iv: "97e63d42a746868d10ab78884fe16a9e"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.769781 28255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:08.777526 28255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:08.780006 28255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:08.783169 28255 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.253:35627
I20260809 00:33:08.783622 28255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
I20260809 00:33:08.783978 28424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.253:35627 every 8 connection(s)
I20260809 00:33:08.784508 28425 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:33:08.785036 28425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: Bootstrap starting.
I20260809 00:33:08.788023 28425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:33:08.788264 28425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: Bootstrap complete.
I20260809 00:33:08.788487 28425 raft_consensus.cc:348] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } }
I20260809 00:33:08.788582 28425 raft_consensus.cc:729] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b71d8be91b3648bd82ab63e06504fe5c, State: Initialized, Role: LEARNER
I20260809 00:33:08.788623 28425 consensus_queue.cc:260] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } }
I20260809 00:33:08.788811 28425 sys_catalog.cc:564] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:08.788813 28428 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } }
I20260809 00:33:08.788990 28428 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: This master's current role is: LEARNER
I20260809 00:33:08.790355 28439 catalog_manager.cc:1259] Loaded cluster ID: 5495401561c54a1887cfc553204f7b88
I20260809 00:33:08.790405 28439 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: loading cluster ID for follower catalog manager: success
I20260809 00:33:08.791407 28439 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: acquiring CA information for follower catalog manager: success
I20260809 00:33:08.791687 28439 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:33:08.805913 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.798728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34788 (local address 127.21.41.253:35627)
0809 00:33:08.798843 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.798845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.798856 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:33:08.798878 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.798879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.798887 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.798960 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.799674 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.800341 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.801371 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.801566 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.801619 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.803084 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.803085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.803086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.803092 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.804632 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.805083 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.805084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.805084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.805136 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.805345 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.805346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.805346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.805446 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.805516 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.805720 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.805817 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for oryx/127.21.41.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.252.keytab.
Entry for principal oryx/127.21.41.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.252.keytab.
I20260809 00:33:08.833616 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.252:43915
--webserver_interface=127.21.41.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.252.keytab
--principal=oryx/127.21.41.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:46427,127.21.41.253:35627,127.21.41.252:43915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:08.917044 28443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:08.917284 28443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:08.917322 28443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:08.918553 28443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:08.918614 28443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:08.918641 28443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:08.918673 28443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:08.918697 28443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:08.920255 28443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:46427,127.21.41.253:35627,127.21.41.252:43915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.252.keytab
--principal=oryx/127.21.41.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.252:43915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.41.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:08.920528 28443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:08.920758 28443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:08.925652 28451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:08.925683 28449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:08.925848 28443 server_base.cc:1034] running on GCE node
W20260809 00:33:08.925923 28448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:08.926192 28443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:08.926541 28443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:08.927870 28443 hybrid_clock.cc:648] HybridClock initialized: now 1786235588927789 us; error 31 us; skew 500 ppm
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:08.930589 28443 init.cc:377] Logged in from keytab as oryx/127.21.41.252@KRBTEST.COM (short username oryx)
I20260809 00:33:08.931102 28443 webserver.cc:460] Webserver started at http://127.21.41.252:33331/ using document root <none> and password file <none>
I20260809 00:33:08.931294 28443 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:08.931339 28443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:08.931432 28443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:08.932469 28443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data/instance:
uuid: "a02a760b5dd940f4adcd6227da071be3"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "aa4a7855b1ee44b995bf202120c18171"
server_key_iv: "7e7e52f45c34cac3443bb9a6f20470ab"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.932778 28443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/wal/instance:
uuid: "a02a760b5dd940f4adcd6227da071be3"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "aa4a7855b1ee44b995bf202120c18171"
server_key_iv: "7e7e52f45c34cac3443bb9a6f20470ab"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.934087 28443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 00:33:08.934971 28458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:08.935189 28443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:33:08.935304 28443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/wal
uuid: "a02a760b5dd940f4adcd6227da071be3"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "aa4a7855b1ee44b995bf202120c18171"
server_key_iv: "7e7e52f45c34cac3443bb9a6f20470ab"
server_key_version: "encryptionkey@0"
I20260809 00:33:08.935384 28443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:08.949028 28443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:08.951659 28443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:08.951834 28443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:08.956034 28443 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.252:43915
I20260809 00:33:08.956106 28510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.252:43915 every 8 connection(s)
I20260809 00:33:08.956461 28443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
I20260809 00:33:08.957561 28511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:08.959097 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 28443
I20260809 00:33:08.959774 28511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.252@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.41.252@KRBTEST.COM
I20260809 00:33:08.969811 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.960718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.21.41.254:46427)
0809 00:33:08.960770 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.960772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.960900 (+ 128us) server_negotiation.cc:407] Connection header received
0809 00:33:08.961018 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.961020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.961037 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.961098 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.961789 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.962550 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.963587 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.963791 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.963880 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.966216 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.966217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.966217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.966225 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.968072 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.968536 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.968538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.968538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.968606 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.969107 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.969108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.969108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.969240 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.969314 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.969579 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.969664 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:33:08.970016 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.960529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59350)
0809 00:33:08.960796 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.960834 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.960955 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.961135 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.961138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.961486 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.961692 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.961697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.962732 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.962733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.963471 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.963473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.963910 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.963911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.963988 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.964647 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.964655 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.966112 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.968155 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.968161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.968163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.968419 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.968921 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.968921 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.968921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.969026 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.969315 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.969323 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.969450 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.969553 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.969777 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":122,"threads_started":1}
I20260809 00:33:08.970451 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.959773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34826 (local address 127.21.41.252:43915)
0809 00:33:08.959978 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.959986 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.960011 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:33:08.960577 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.960586 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.960654 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.960833 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.961507 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.962845 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.963563 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.963874 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.963988 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.966830 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.966838 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.966838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.966845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.968600 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.968918 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.968920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.968920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.969029 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.969250 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.969251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.969252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.969456 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.969596 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.969683 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.970302 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
I20260809 00:33:08.970695 28511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.252@KRBTEST.COM for oryx/127.21.41.253@KRBTEST.COM
I20260809 00:33:08.976856 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.971072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35627 (local address 127.0.0.1:34794)
0809 00:33:08.971121 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.971131 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.971166 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.971294 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.971295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.971404 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.971440 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.971441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.972049 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.972049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.972532 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.972533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.972865 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.972866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.972915 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.973518 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.973523 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.974728 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.976051 (+ 1323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.976052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.976052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.976232 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.976388 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.976388 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.976389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.976431 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.976639 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.976639 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.976672 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.976725 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.976791 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 00:33:08.976863 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.971143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.21.41.253:35627)
0809 00:33:08.971180 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.971182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.971190 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:33:08.971236 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.971237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.971248 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.971294 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.971499 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.971983 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.972600 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.972791 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.972841 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.974806 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.974808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.974808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.974814 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.975979 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.976297 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.976298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.976298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.976344 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.976489 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.976490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.976490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.976582 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.976630 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.976740 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.976802 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:33:08.977345 28511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:33:08 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235588, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.252@KRBTEST.COM for oryx/127.21.41.252@KRBTEST.COM
I20260809 00:33:08.983104 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.977611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:43915 (local address 127.0.0.1:34838)
0809 00:33:08.977661 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.977673 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.977719 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.977829 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.977829 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.977932 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.977957 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.977958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.978581 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.978582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.979052 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.979053 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.979400 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.979401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.979436 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.979774 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.979778 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:08.980866 (+ 1088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:08.982246 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.982247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.982247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.982470 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.982628 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:08.982629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:08.982629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:08.982678 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:08.982854 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:08.982855 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:08.982896 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 00:33:08.982954 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:33:08.983024 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 00:33:08.983152 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.977663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34838 (local address 127.21.41.252:43915)
0809 00:33:08.977698 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.977699 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.977706 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:33:08.977776 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.977776 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.977783 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.977827 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.977995 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.978519 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.979125 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.979337 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.979390 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.980937 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:08.980938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:08.980938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:08.980943 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:08.982173 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.982532 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.982532 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.982532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.982574 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:08.982722 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:08.982722 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:08.982722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:08.982806 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:08.982847 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:08.982968 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 00:33:08.983064 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":17}
I20260809 00:33:08.984508 28511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: Bootstrap starting.
I20260809 00:33:08.985342 28511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:08.985628 28511 log.cc:824] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:08.986442 28511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: No bootstrap required, opened a new log
I20260809 00:33:08.987927 28511 raft_consensus.cc:348] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } }
I20260809 00:33:08.988085 28511 raft_consensus.cc:374] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:08.988166 28511 raft_consensus.cc:729] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02a760b5dd940f4adcd6227da071be3, State: Initialized, Role: FOLLOWER
I20260809 00:33:08.988341 28511 consensus_queue.cc:260] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } }
I20260809 00:33:08.988804 28511 sys_catalog.cc:564] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:08.988754 28516 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } } }
I20260809 00:33:08.988879 28516 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:33:08.990866 28527 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:33:08.990928 28527 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:33:08.993142 28443 master_runner.cc:428] Detected that this master a02a760b5dd940f4adcd6227da071be3 is joining an existing cluster
I20260809 00:33:08.993196 28443 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.41.252:43915
I20260809 00:33:09.003428 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.994527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.21.41.254:46427)
0809 00:33:08.994604 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.994606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.994850 (+ 244us) server_negotiation.cc:407] Connection header received
0809 00:33:08.994909 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.994910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.994922 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.994952 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.995472 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.996292 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.999321 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.999674 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.999750 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.000840 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.000841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.000842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.000849 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.002189 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.002579 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.002580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.002581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.002644 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.002867 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.002868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.002868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.002987 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.003061 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.003260 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.003333 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 00:33:09.003445 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.994400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59366)
0809 00:33:08.994768 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.994780 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.994843 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.995023 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.995023 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.995283 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.995371 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.995374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.998660 (+ 3286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.998660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.999231 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.999233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.999815 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.999817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.999879 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.000375 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.000380 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.000720 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.002277 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.002278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.002278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.002490 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.002729 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.002730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.002730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.002790 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.003063 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.003065 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.003125 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.003219 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.003330 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":2,"spinlock_wait_cycles":9984,"thread_start_us":101,"threads_started":1}
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235587, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.254@KRBTEST.COM for oryx/127.21.41.252@KRBTEST.COM
I20260809 00:33:09.021924 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 27851
I20260809 00:33:09.027547 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:38675
--local_ip_for_outbound_sockets=127.21.41.193
--tserver_master_addrs=127.21.41.254:46427
--webserver_port=44957
--webserver_interface=127.21.41.193
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab
--principal=oryx/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.41.254:46427,127.21.41.253:35627
--tserver_master_addrs=127.21.41.254:46427,127.21.41.253:35627,127.21.41.252:43915 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:33:09.030232 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.994484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34798 (local address 127.21.41.253:35627)
0809 00:33:08.994571 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.994573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.994880 (+ 307us) server_negotiation.cc:407] Connection header received
0809 00:33:08.994906 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.994907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.994918 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.994989 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.995603 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.996383 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.997541 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.997850 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.997887 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.026909 (+ 29022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.026912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.026912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.026922 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.028680 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.029207 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.029209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.029210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.029279 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.029476 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.029477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.029478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.029632 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.029690 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.030025 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.030125 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 00:33:09.031486 28535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.994401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35627 (local address 127.0.0.1:34798)
0809 00:33:08.994768 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.994780 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.994843 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.994998 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.994999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.995416 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.995486 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.995488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.996518 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.996519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.997436 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.997439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.997957 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.997957 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.998009 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:08.999855 (+ 1846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:08.999860 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.026783 (+ 26923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.028798 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.028799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.028800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.029113 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.029351 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.029352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.029352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.029412 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.029731 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.029732 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.029783 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.029849 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.031364 (+ 1515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":31,"thread_start_us":84,"threads_started":1}
I20260809 00:33:09.032105 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.005802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:43915 (local address 127.0.0.1:34854)
0809 00:33:09.005861 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.005873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.005922 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.006190 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.006192 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.006413 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:09.006458 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.006460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.007214 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.007215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.008083 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.008086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.008470 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.008471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.008543 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.008914 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.008919 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.010502 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.030824 (+ 20322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.030826 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.030827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.031085 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.031311 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.031312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.031313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.031379 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.031686 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.031687 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.031756 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.031812 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.031944 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 00:33:09.032598 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.005867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34854 (local address 127.21.41.252:43915)
0809 00:33:09.006063 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.006064 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.006071 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:33:09.006101 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.006102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.006110 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.006165 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.006536 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.007133 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.008168 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.008393 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.008451 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.010612 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.010614 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.010614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.010620 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.030698 (+ 20078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.031170 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.031171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.031172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.031237 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.031448 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.031449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.031450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.031597 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.032298 (+ 701us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.032359 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.032452 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":129,"threads_started":1}
I20260809 00:33:09.033003 28533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.994647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:43915 (local address 127.0.0.1:34844)
0809 00:33:08.995088 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:08.995098 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:08.995157 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:08.995322 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:08.995323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:08.995523 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:08.995595 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.995597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.996483 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.996484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.997472 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:08.997474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.997947 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.997948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:08.998013 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.027643 (+ 29630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.027649 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.029292 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.031966 (+ 2674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.031967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.031967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.032189 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.032397 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.032398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.032398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.032443 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.032689 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.032690 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.032732 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.032800 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.032895 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":34,"thread_start_us":59,"threads_started":1}
I20260809 00:33:09.033192 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:08.994676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34844 (local address 127.21.41.252:43915)
0809 00:33:08.994713 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:08.994714 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:08.995140 (+ 426us) server_negotiation.cc:407] Connection header received
0809 00:33:08.995235 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:08.995236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:08.995245 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:08.995319 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:08.995670 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.996380 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.997549 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:08.997823 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:08.997859 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.029398 (+ 31539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.029400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.029400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.029407 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.031255 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.032290 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.032291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.032291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.032336 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.032503 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.032504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.032504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.032613 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.032648 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.032989 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.033067 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:33:09.033146 27789 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.41.252:43915: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a02a760b5dd940f4adcd6227da071be3"
member_type: NON_VOTER
last_known_addr {
host: "127.21.41.252"
port: 43915
}
attrs {
promote: true
}
}
dest_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582"
cas_config_opid_index: 5
I20260809 00:33:09.033432 27789 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } }
I20260809 00:33:09.033856 28350 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a02a760b5dd940f4adcd6227da071be3 (127.21.41.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.034881 28350 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.41.252:43915
I20260809 00:33:09.034957 28350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.035028 28350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:09.035468 28485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:09.036408 28485 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 FOLLOWER]: Refusing update from remote peer d4914f52a2dc4d7eaf6c0d04db25f582: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 00:33:09.036739 28351 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:09.037113 28443 master.cc:537] Master@127.21.41.252:43915 shutting down...
I20260809 00:33:09.039646 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.033914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35627 (local address 127.0.0.1:34804)
0809 00:33:09.034048 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.034060 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.034102 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.034379 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.034380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.034382 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:09.034433 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.034435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.035427 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.035429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.036805 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.036809 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.039304 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.039306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.039418 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.039445 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.039506 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.039556 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260809 00:33:09.039831 28485 raft_consensus.cc:858] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } }
W20260809 00:33:09.039914 28485 raft_consensus.cc:1565] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } }
I20260809 00:33:09.039973 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.034012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34804 (local address 127.21.41.253:35627)
0809 00:33:09.034172 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.034175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.034182 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:33:09.034205 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.034206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.034231 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.034293 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:09.034515 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.035323 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.036910 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.037810 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.037884 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.037965 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.039833 (+ 1868us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.039883 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260809 00:33:09.040179 28391 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 LEARNER]: Refusing update from remote peer d4914f52a2dc4d7eaf6c0d04db25f582: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 00:33:09.040397 28351 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 00:33:09.040442 28485 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b71d8be91b3648bd82ab63e06504fe5c (127.21.41.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } }
I20260809 00:33:09.040931 28391 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a02a760b5dd940f4adcd6227da071be3 (127.21.41.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.041951 28428 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4914f52a2dc4d7eaf6c0d04db25f582. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.042040 28428 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: This master's current role is: LEARNER
I20260809 00:33:09.042037 28516 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4914f52a2dc4d7eaf6c0d04db25f582. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } }
I20260809 00:33:09.042073 28428 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.042119 28428 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: This master's current role is: LEARNER
I20260809 00:33:09.042122 28516 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 00:33:09.042167 28516 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } }
I20260809 00:33:09.042219 28516 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 00:33:09.042258 28516 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } }
I20260809 00:33:09.042313 28516 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 00:33:09.043068 28351 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: NON_VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: true } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.043161 28351 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: This master's current role is: LEADER
W20260809 00:33:09.043596 27774 consensus_peers.cc:487] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 -> Peer a02a760b5dd940f4adcd6227da071be3 (127.21.41.252:43915): Couldn't send request to peer a02a760b5dd940f4adcd6227da071be3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 00:33:09.044180 28350 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582: attempting to promote NON_VOTER b71d8be91b3648bd82ab63e06504fe5c to VOTER
I20260809 00:33:09.044379 28350 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } }
I20260809 00:33:09.044484 28538 mvcc.cc:205] Tried to move back new op lower bound from 7316420968553525248 to 7316420968469569536. Current Snapshot: MvccSnapshot[applied={T|T < 7316420968480645120}]
I20260809 00:33:09.044548 28443 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260809 00:33:09.044637 28443 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260809 00:33:09.044684 28443 tablet_replica.cc:330] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: stopping tablet replica
I20260809 00:33:09.044968 28391 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 LEARNER]: Refusing update from remote peer d4914f52a2dc4d7eaf6c0d04db25f582: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 00:33:09.045133 28351 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 00:33:09.045627 27774 consensus_peers.cc:487] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 -> Peer a02a760b5dd940f4adcd6227da071be3 (127.21.41.252:43915): Couldn't send request to peer a02a760b5dd940f4adcd6227da071be3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 00:33:09.046240 28547 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b71d8be91b3648bd82ab63e06504fe5c (127.21.41.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.046473 28391 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b71d8be91b3648bd82ab63e06504fe5c (127.21.41.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.047403 28351 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.047482 28351 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:09.047570 28547 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.047628 28547 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:09.048462 28543 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.048561 28543 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:33:09.060045 28443 master.cc:552] Master@127.21.41.252:43915 shutdown complete.
I20260809 00:33:09.060114 28443 master_runner.cc:305] Clearing existing system tablet
I20260809 00:33:09.060248 28443 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:09.060297 28443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:09.061065 28443 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:33:09.061642 28550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.061755 28443 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:09.061800 28443 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.061821 28443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/wal
uuid: "a02a760b5dd940f4adcd6227da071be3"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "aa4a7855b1ee44b995bf202120c18171"
server_key_iv: "7e7e52f45c34cac3443bb9a6f20470ab"
server_key_version: "encryptionkey@0"
I20260809 00:33:09.062633 28443 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:09.064317 28443 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 00:33:09.064386 28443 log.cc:1196] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 00:33:09.064627 28443 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: Deleting consensus metadata
I20260809 00:33:09.064754 28443 master_runner.cc:315] Copying system tablet from 127.21.41.254:46427
I20260809 00:33:09.065384 28443 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: tablet copy: Beginning tablet copy session from remote peer at address 127.21.41.254:46427
I20260809 00:33:09.072935 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.065701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59376)
0809 00:33:09.065825 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.065835 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.065895 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.066064 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.066065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.066262 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:09.066361 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.066363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.067305 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.067306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.068123 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.068150 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.068631 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.068632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.068701 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.069222 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.069227 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.069664 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.071731 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.071732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.071732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.071959 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.072186 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.072187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.072187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.072258 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.072587 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.072588 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.072639 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.072722 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.072831 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260809 00:33:09.072943 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.065779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.21.41.254:46427)
0809 00:33:09.065831 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.065833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.065882 (+ 49us) server_negotiation.cc:407] Connection header received
0809 00:33:09.065957 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.065957 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.065968 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.066045 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.066437 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.067205 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.068243 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.068537 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.068613 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.069758 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.069760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.069760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.069768 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.071634 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.072043 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.072044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.072044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.072106 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.072372 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.072373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.072373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.072507 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.072577 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.072770 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.072850 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:33:09.073653 27809 tablet_copy_service.cc:140] P d4914f52a2dc4d7eaf6c0d04db25f582: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a02a760b5dd940f4adcd6227da071be3 ({username='oryx', principal='oryx/127.21.41.252@KRBTEST.COM'} at 127.0.0.1:59376)
I20260809 00:33:09.073719 27809 tablet_copy_service.cc:161] P d4914f52a2dc4d7eaf6c0d04db25f582: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a02a760b5dd940f4adcd6227da071be3 at {username='oryx', principal='oryx/127.21.41.252@KRBTEST.COM'} at 127.0.0.1:59376: session id = a02a760b5dd940f4adcd6227da071be3-00000000000000000000000000000000
I20260809 00:33:09.074295 27809 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 00:33:09.074923 28443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 00:33:09.076213 28443 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: tablet copy: Starting download of 0 data blocks...
I20260809 00:33:09.076345 28443 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: tablet copy: Starting download of 1 WAL segments...
I20260809 00:33:09.077201 28443 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 00:33:09.078296 27809 tablet_copy_service.cc:342] P d4914f52a2dc4d7eaf6c0d04db25f582: Request end of tablet copy session a02a760b5dd940f4adcd6227da071be3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.41.252@KRBTEST.COM'} at 127.0.0.1:59376
I20260809 00:33:09.078394 27809 tablet_copy_service.cc:433] P d4914f52a2dc4d7eaf6c0d04db25f582: ending tablet copy session a02a760b5dd940f4adcd6227da071be3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a02a760b5dd940f4adcd6227da071be3
I20260809 00:33:09.081256 28443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:09.082508 28557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:09.082556 28560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:09.082798 28558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:09.082881 28443 server_base.cc:1034] running on GCE node
I20260809 00:33:09.082969 28443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:09.083096 28443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:09.084221 28443 hybrid_clock.cc:648] HybridClock initialized: now 1786235589084211 us; error 26 us; skew 500 ppm
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:09.085752 28443 init.cc:377] Logged in from keytab as oryx/127.21.41.252@KRBTEST.COM (short username oryx)
I20260809 00:33:09.086009 28443 webserver.cc:460] Webserver started at http://127.21.41.252:42973/ using document root <none> and password file <none>
I20260809 00:33:09.086113 28443 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:09.086139 28443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:09.086902 28443 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:33:09.087504 28567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.087682 28443 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.087736 28443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/wal
uuid: "a02a760b5dd940f4adcd6227da071be3"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "aa4a7855b1ee44b995bf202120c18171"
server_key_iv: "7e7e52f45c34cac3443bb9a6f20470ab"
server_key_version: "encryptionkey@0"
I20260809 00:33:09.087788 28443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:09.111768 28443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:09.114897 28443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:09.119733 28443 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.252:43915
I20260809 00:33:09.119781 28619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.252:43915 every 8 connection(s)
I20260809 00:33:09.120246 28443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
I20260809 00:33:09.121341 28620 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:33:09.121924 28620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: Bootstrap starting.
I20260809 00:33:09.124567 28620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:33:09.124796 28620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: Bootstrap complete.
I20260809 00:33:09.124995 28620 raft_consensus.cc:348] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } }
I20260809 00:33:09.125087 28620 raft_consensus.cc:729] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a02a760b5dd940f4adcd6227da071be3, State: Initialized, Role: LEARNER
I20260809 00:33:09.125124 28620 consensus_queue.cc:260] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } }
I20260809 00:33:09.125348 28620 sys_catalog.cc:564] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:09.125313 28623 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.125511 28623 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: This master's current role is: LEARNER
I20260809 00:33:09.127144 28634 catalog_manager.cc:1259] Loaded cluster ID: 5495401561c54a1887cfc553204f7b88
I20260809 00:33:09.127243 28634 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: loading cluster ID for follower catalog manager: success
I20260809 00:33:09.128633 28634 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: acquiring CA information for follower catalog manager: success
I20260809 00:33:09.129032 28634 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 00:33:09.137755 28537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:09.137981 28537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:09.138020 28537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:09.139261 28537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:09.139329 28537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:09.139375 28537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:33:09.140867 28537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab
--principal=oryx/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:38675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=44957
--tserver_master_addrs=127.21.41.254:46427,127.21.41.253:35627,127.21.41.252:43915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:09.141134 28537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:09.141433 28537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:09.147853 28639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:09.148160 28537 server_base.cc:1034] running on GCE node
W20260809 00:33:09.147957 28642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:09.147876 28640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:09.148479 28537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:09.148731 28537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:09.149899 28537 hybrid_clock.cc:648] HybridClock initialized: now 1786235589149877 us; error 41 us; skew 500 ppm
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:09.152439 28537 init.cc:377] Logged in from keytab as oryx/127.21.41.193@KRBTEST.COM (short username oryx)
I20260809 00:33:09.152925 28537 webserver.cc:460] Webserver started at http://127.21.41.193:44957/ using document root <none> and password file <none>
I20260809 00:33:09.153175 28537 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:09.153236 28537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:09.154675 28537 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:33:09.155324 28649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.155498 28537 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:09.155615 28537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "660fd2a7b01b4e15a201d53f7fe61e58"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "8109878b4a47237e01dfffe6f36b971a"
server_key_iv: "95b8f721cd766c33bb7833fd43b4942c"
server_key_version: "encryptionkey@0"
I20260809 00:33:09.155977 28537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:09.166567 28537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:09.168366 28537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:09.168532 28537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:09.168825 28537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:09.169140 28537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:09.169194 28537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.169229 28537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:09.169256 28537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.175349 28537 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:38675
I20260809 00:33:09.175455 28762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:38675 every 8 connection(s)
I20260809 00:33:09.175801 28537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.252@KRBTEST.COM
I20260809 00:33:09.183924 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 28537
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.253@KRBTEST.COM
I20260809 00:33:09.184108 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 27985
I20260809 00:33:09.189160 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.177014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:36779 (local address 127.21.41.254:46427)
0809 00:33:09.177058 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.177060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.177069 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:09.178021 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.178023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.178035 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.178087 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.178904 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.179417 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.180863 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.181171 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.181211 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.183376 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.183378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.183378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.183384 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.187372 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.187935 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.187936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.187937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.187997 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.188262 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.188263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.188263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.188434 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.188491 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.188900 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.188989 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:33:09.189234 28768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.176552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:43915 (local address 127.21.41.193:42099)
0809 00:33:09.176730 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.176794 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.177825 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.178233 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.178233 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.178592 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:09.178805 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.178810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.179993 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.179994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.180723 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.180726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.181249 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.181250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.181303 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.182017 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.182023 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.183785 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.186150 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.186158 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.186160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.187784 (+ 1624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.188009 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.188010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.188011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.188145 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.188460 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.188467 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.188704 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.188790 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.188976 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":38,"thread_start_us":87,"threads_started":1}
I20260809 00:33:09.189301 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.176543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:35293 (local address 127.21.41.253:35627)
0809 00:33:09.176594 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.176597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.176889 (+ 292us) server_negotiation.cc:407] Connection header received
0809 00:33:09.177952 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.177953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.177967 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.178021 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.178918 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.179701 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.180813 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.181030 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.181111 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.185143 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.185145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.185145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.185153 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.188125 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.188493 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.188495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.188495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.188549 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.188748 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.188749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.188750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.188884 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.188926 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.189118 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.189190 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:33:09.189575 28767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.176543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:42099 (local address 127.21.41.252:43915)
0809 00:33:09.176665 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.176668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.176889 (+ 221us) server_negotiation.cc:407] Connection header received
0809 00:33:09.178121 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.178122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.178137 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.178221 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.178918 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.179905 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.180845 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.181153 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.181229 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.183890 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.183891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.183891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.183899 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.186057 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.187872 (+ 1815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.187874 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.187874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.187939 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.188217 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.188218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.188218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.188371 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.188433 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.189373 (+ 940us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.189471 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260809 00:33:09.189769 28769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.176552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35627 (local address 127.21.41.193:35293)
0809 00:33:09.176721 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.176799 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.177825 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.178093 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.178097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.178600 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:09.178805 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.178810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.179782 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.179783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.180724 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.180727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.181117 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.181118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.181258 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.182155 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.182159 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.185043 (+ 2884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.188220 (+ 3177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.188221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.188221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.188411 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.188619 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.188619 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.188620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.188674 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.188977 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.188977 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.189012 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.189545 (+ 533us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.189638 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":11,"thread_start_us":91,"threads_started":1}
I20260809 00:33:09.190173 28770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.176845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.193:36779)
0809 00:33:09.176952 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.176962 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.177841 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.178126 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.178127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.178562 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:09.178804 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.178810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.179553 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.179554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.180771 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.180772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.181325 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.181326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.181373 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.181773 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.181782 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.183283 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.187444 (+ 4161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.187445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.187446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.187842 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.188127 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.188128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.188129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.188192 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.188591 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.188592 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.188645 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.189981 (+ 1336us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.190075 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":33,"thread_start_us":56,"threads_started":1}
I20260809 00:33:09.190371 28763 heartbeater.cc:347] Connected to a master server at 127.21.41.252:43915
I20260809 00:33:09.190466 28763 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:09.190538 28765 heartbeater.cc:347] Connected to a master server at 127.21.41.253:35627
I20260809 00:33:09.190604 28765 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:09.190651 28763 heartbeater.cc:511] Master 127.21.41.252:43915 requested a full tablet report, sending...
I20260809 00:33:09.190676 28765 heartbeater.cc:511] Master 127.21.41.253:35627 requested a full tablet report, sending...
I20260809 00:33:09.191184 28380 ts_manager.cc:194] Registered new tserver with Master: 660fd2a7b01b4e15a201d53f7fe61e58 (127.21.41.193:38675)
I20260809 00:33:09.191296 28584 ts_manager.cc:194] Registered new tserver with Master: 660fd2a7b01b4e15a201d53f7fe61e58 (127.21.41.193:38675)
I20260809 00:33:09.191936 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:32937
--local_ip_for_outbound_sockets=127.21.41.194
--tserver_master_addrs=127.21.41.254:46427
--webserver_port=33049
--webserver_interface=127.21.41.194
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab
--principal=oryx/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.41.254:46427,127.21.41.253:35627
--tserver_master_addrs=127.21.41.254:46427,127.21.41.253:35627,127.21.41.252:43915 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:33:09.191955 28764 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46427
I20260809 00:33:09.192035 28764 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:09.192111 28764 heartbeater.cc:511] Master 127.21.41.254:46427 requested a full tablet report, sending...
I20260809 00:33:09.192368 27789 ts_manager.cc:194] Re-registered known tserver with Master: 660fd2a7b01b4e15a201d53f7fe61e58 (127.21.41.193:38675)
I20260809 00:33:09.192756 27789 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:36779
W20260809 00:33:09.283100 28771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:09.283349 28771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:09.283393 28771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:09.284688 28771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:09.284760 28771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:09.284811 28771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:33:09.286566 28771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab
--principal=oryx/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:32937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=33049
--tserver_master_addrs=127.21.41.254:46427,127.21.41.253:35627,127.21.41.252:43915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:09.286878 28771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:09.287137 28771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:09.293079 28776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:09.293097 28777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:09.293130 28779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:09.293270 28771 server_base.cc:1034] running on GCE node
I20260809 00:33:09.293597 28771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:09.293843 28771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:09.294984 28771 hybrid_clock.cc:648] HybridClock initialized: now 1786235589294964 us; error 34 us; skew 500 ppm
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:09.297463 28771 init.cc:377] Logged in from keytab as oryx/127.21.41.194@KRBTEST.COM (short username oryx)
I20260809 00:33:09.297966 28771 webserver.cc:460] Webserver started at http://127.21.41.194:33049/ using document root <none> and password file <none>
I20260809 00:33:09.298213 28771 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:09.298298 28771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:09.299644 28771 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:09.300237 28786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.300420 28771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:09.300525 28771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "873a29f3b7e0771dbbb8f4ef40b03a9d"
server_key_iv: "92fdea870c3f253ed47cd7a2439452d7"
server_key_version: "encryptionkey@0"
I20260809 00:33:09.300894 28771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:09.310297 28771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:09.312042 28771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:09.312201 28771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:09.312426 28771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:09.312798 28771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:09.312852 28771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.312893 28771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:09.312920 28771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.318967 28771 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:32937
I20260809 00:33:09.319013 28900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:32937 every 8 connection(s)
I20260809 00:33:09.319339 28771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.252@KRBTEST.COM
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.253@KRBTEST.COM
I20260809 00:33:09.327409 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 28771
I20260809 00:33:09.327574 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 28119
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:09.330751 28767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.320443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:56073 (local address 127.21.41.252:43915)
0809 00:33:09.320498 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.320500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.320509 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:09.321361 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.321363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.321379 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.321410 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.322328 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.323070 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.323923 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.324200 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.324265 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.327086 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.327088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.327088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.327096 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.328758 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.329533 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.329534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.329535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.329584 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.329890 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.329891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.329891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.330043 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.330092 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.330536 (+ 444us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.330633 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:33:09.331341 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.320329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:43915 (local address 127.21.41.194:56073)
0809 00:33:09.320448 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.320457 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.321260 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.321511 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.321511 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.321950 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:09.322263 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.322265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.323148 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.323149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.323833 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.323835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.324278 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.324279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.324376 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.324964 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.324991 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.326953 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.328848 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.328856 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.328859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.329449 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.329660 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.329661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.329661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.329795 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.330148 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.330155 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.330368 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.330885 (+ 517us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.331068 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":3,"thread_start_us":56,"threads_started":1}
I20260809 00:33:09.331457 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.320046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:49381 (local address 127.21.41.253:35627)
0809 00:33:09.320097 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.320099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.320336 (+ 237us) server_negotiation.cc:407] Connection header received
0809 00:33:09.321358 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.321360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.321374 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.321433 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.322240 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.323035 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.324087 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.324376 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.324425 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.328274 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.328275 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.328276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.328284 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.330024 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.330468 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.330469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.330469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.330526 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.330714 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.330715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.330716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.330854 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.330898 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.331227 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.331315 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:33:09.332170 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.320039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35627 (local address 127.21.41.194:49381)
0809 00:33:09.320229 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.320252 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.321239 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.321501 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.321505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.321939 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:09.322141 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.322145 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.323120 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.323121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.324004 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.324006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.324444 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.324445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.324506 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.325100 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.325104 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.328176 (+ 3072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.330118 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.330119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.330120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.330385 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.330594 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.330595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.330595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.330652 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.331054 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.331055 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.331101 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.331889 (+ 788us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.332004 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":56,"thread_start_us":95,"threads_started":1}
I20260809 00:33:09.332413 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.320054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:56931 (local address 127.21.41.254:46427)
0809 00:33:09.320111 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.320113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.320333 (+ 220us) server_negotiation.cc:407] Connection header received
0809 00:33:09.321362 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.321363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.321376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.321410 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.322247 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.322808 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.324471 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.324739 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.324804 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.329479 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.329481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.329481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.329491 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.331206 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.331626 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.331628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.331628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.331689 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.331898 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.331899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.331899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.332015 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.332090 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.332244 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.332321 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:33:09.332422 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.320039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.194:56931)
0809 00:33:09.320205 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.320249 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.321260 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.321512 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.321512 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.321943 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:09.322142 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.322148 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.322908 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.322909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.324004 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.324006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.324861 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.324862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.324932 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.325353 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.325358 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.329371 (+ 4013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.331298 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.331299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.331299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.331505 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.331760 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.331760 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.331761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.331819 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.332082 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.332083 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.332124 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.332214 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.332321 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":11,"thread_start_us":93,"threads_started":1}
I20260809 00:33:09.332893 28903 heartbeater.cc:347] Connected to a master server at 127.21.41.253:35627
I20260809 00:33:09.332894 28902 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46427
I20260809 00:33:09.333015 28902 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:09.333015 28903 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:09.333245 28903 heartbeater.cc:511] Master 127.21.41.253:35627 requested a full tablet report, sending...
I20260809 00:33:09.333245 28902 heartbeater.cc:511] Master 127.21.41.254:46427 requested a full tablet report, sending...
I20260809 00:33:09.333701 27789 ts_manager.cc:194] Re-registered known tserver with Master: cf1f7a8afa5a42a7853b7ad1c2ee7855 (127.21.41.194:32937)
I20260809 00:33:09.333781 28380 ts_manager.cc:194] Registered new tserver with Master: cf1f7a8afa5a42a7853b7ad1c2ee7855 (127.21.41.194:32937)
I20260809 00:33:09.334224 27789 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:56931
I20260809 00:33:09.335480 28901 heartbeater.cc:347] Connected to a master server at 127.21.41.252:43915
I20260809 00:33:09.335526 28901 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:09.335621 28901 heartbeater.cc:511] Master 127.21.41.252:43915 requested a full tablet report, sending...
I20260809 00:33:09.335980 28584 ts_manager.cc:194] Registered new tserver with Master: cf1f7a8afa5a42a7853b7ad1c2ee7855 (127.21.41.194:32937)
I20260809 00:33:09.335978 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:42291
--local_ip_for_outbound_sockets=127.21.41.195
--tserver_master_addrs=127.21.41.254:46427
--webserver_port=41753
--webserver_interface=127.21.41.195
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab
--principal=oryx/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.41.254:46427,127.21.41.253:35627
--tserver_master_addrs=127.21.41.254:46427,127.21.41.253:35627,127.21.41.252:43915 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:09.419137 28908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:09.419325 28908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:09.419348 28908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:09.420436 28908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:09.420475 28908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:09.420499 28908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:09.421880 28908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab
--principal=oryx/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:42291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=41753
--tserver_master_addrs=127.21.41.254:46427,127.21.41.253:35627,127.21.41.252:43915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:09.422065 28908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:09.422246 28908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:09.427425 28916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:09.427424 28914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:09.427454 28913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:09.427767 28908 server_base.cc:1034] running on GCE node
I20260809 00:33:09.427991 28908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:09.428223 28908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:09.429392 28908 hybrid_clock.cc:648] HybridClock initialized: now 1786235589429385 us; error 36 us; skew 500 ppm
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:09.431905 28908 init.cc:377] Logged in from keytab as oryx/127.21.41.195@KRBTEST.COM (short username oryx)
I20260809 00:33:09.432343 28908 webserver.cc:460] Webserver started at http://127.21.41.195:41753/ using document root <none> and password file <none>
I20260809 00:33:09.432585 28908 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:09.432665 28908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:09.433981 28908 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:09.434621 28924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.434831 28908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:33:09.434940 28908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "53ea8b2562624f53a4c81f0d3ee0be50"
format_stamp: "Formatted at 2026-08-09 00:33:08 on dist-test-slave-zpfg"
server_key: "928ff3cf43eb910a816208f4e1d9b6c3"
server_key_iv: "443c6127795e93a60fa51321cbad66ca"
server_key_version: "encryptionkey@0"
I20260809 00:33:09.435292 28908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:09.442498 28908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:09.444722 28908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:09.444886 28908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:09.445104 28908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:09.445407 28908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:09.445459 28908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.445501 28908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:09.445528 28908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:09.451344 28908 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:42291
I20260809 00:33:09.451436 29037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:42291 every 8 connection(s)
I20260809 00:33:09.451750 28908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.252@KRBTEST.COM
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.253@KRBTEST.COM
Aug 09 00:33:09 dist-test-slave-zpfg krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235589, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:09.461042 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 28908
I20260809 00:33:09.463739 28767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.452574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:50409 (local address 127.21.41.252:43915)
0809 00:33:09.452737 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.452739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.452944 (+ 205us) server_negotiation.cc:407] Connection header received
0809 00:33:09.454016 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.454017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.454031 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.454102 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.454906 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.455412 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.457047 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.457345 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.457414 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.460062 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.460062 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.460063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.460070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.461913 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.462452 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.462453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.462454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.462516 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.462949 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.462950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.462950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.463100 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.463154 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.463531 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.463620 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260809 00:33:09.464368 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.452549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:49607 (local address 127.21.41.253:35627)
0809 00:33:09.452732 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.452734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.452955 (+ 221us) server_negotiation.cc:407] Connection header received
0809 00:33:09.454024 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.454025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.454038 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.454072 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.454915 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.455712 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.457042 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.457344 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.457414 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.460813 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.460815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.460815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.460823 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.463153 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.463571 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.463572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.463572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.463629 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.463821 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.463821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.463822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.463946 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.463988 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.464198 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.464285 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260809 00:33:09.464414 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.452566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:43915 (local address 127.21.41.195:50409)
0809 00:33:09.452819 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.452862 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.453903 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.454155 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.454159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.454594 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:09.454807 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.454811 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.455518 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.455519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.456949 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.456952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.457432 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.457433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.457564 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.458167 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.458177 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.459955 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.462007 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.462014 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.462017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.462337 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.462714 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.462714 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.462715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.462861 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.463187 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.463194 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.463359 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.463987 (+ 628us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.464185 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":59,"thread_start_us":99,"threads_started":1}
I20260809 00:33:09.464854 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.452565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35627 (local address 127.21.41.195:49607)
0809 00:33:09.452867 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.452877 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.453902 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.454154 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.454159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.454594 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:09.454808 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.454814 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.455788 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.455789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.456949 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.456952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.457804 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.457805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.457868 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.458439 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.458444 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.460707 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.463229 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.463229 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.463230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.463490 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.463700 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.463700 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.463701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.463751 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.464034 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.464035 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.464071 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.464612 (+ 541us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.464750 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":14,"thread_start_us":116,"threads_started":1}
I20260809 00:33:09.465092 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.452979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:40015 (local address 127.21.41.254:46427)
0809 00:33:09.453022 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.453024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.453154 (+ 130us) server_negotiation.cc:407] Connection header received
0809 00:33:09.454026 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.454027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.454039 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.454072 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.454915 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.455701 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.457796 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.458105 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.458210 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.461164 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.461166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.461166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.461175 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.463236 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.463703 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.463717 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.463717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.463781 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.464170 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.464171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.464172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.464324 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.464373 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.464910 (+ 537us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.465001 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:33:09.465190 29038 heartbeater.cc:347] Connected to a master server at 127.21.41.252:43915
I20260809 00:33:09.465304 29038 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:09.465502 29038 heartbeater.cc:511] Master 127.21.41.252:43915 requested a full tablet report, sending...
I20260809 00:33:09.465637 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.452900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.195:40015)
0809 00:33:09.453088 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.453097 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.453915 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.454154 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.454159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.454594 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:09.454808 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.454814 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.457057 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.457058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.457688 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.457690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.458187 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.458187 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.458248 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.459077 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:09.459081 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:09.461014 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:09.463330 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.463331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.463332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.463615 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.464028 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:09.464029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:09.464029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:09.464092 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:09.464552 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:09.464553 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:09.464598 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.464810 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.465346 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":3,"thread_start_us":64,"threads_started":1}
I20260809 00:33:09.465646 29040 heartbeater.cc:347] Connected to a master server at 127.21.41.253:35627
I20260809 00:33:09.465718 29040 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:09.465783 29040 heartbeater.cc:511] Master 127.21.41.253:35627 requested a full tablet report, sending...
I20260809 00:33:09.465965 28584 ts_manager.cc:194] Registered new tserver with Master: 53ea8b2562624f53a4c81f0d3ee0be50 (127.21.41.195:42291)
I20260809 00:33:09.466075 28380 ts_manager.cc:194] Registered new tserver with Master: 53ea8b2562624f53a4c81f0d3ee0be50 (127.21.41.195:42291)
I20260809 00:33:09.466262 29039 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46427
I20260809 00:33:09.466320 29039 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:09.466405 29039 heartbeater.cc:511] Master 127.21.41.254:46427 requested a full tablet report, sending...
I20260809 00:33:09.466629 27789 ts_manager.cc:194] Re-registered known tserver with Master: 53ea8b2562624f53a4c81f0d3ee0be50 (127.21.41.195:42291)
I20260809 00:33:09.467144 27789 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:40015
I20260809 00:33:09.474089 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.462612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.21.41.254:46427)
0809 00:33:09.465201 (+ 2589us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.465203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.465210 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:33:09.465229 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.465229 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.465238 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.465292 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.466873 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.467544 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.468567 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.468820 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.468888 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.470985 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.470986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.470987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.470994 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.472908 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.473316 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.473317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.473318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.473374 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.473564 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.473566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.473566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.473704 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.473817 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.473912 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.473982 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2568,"thread_start_us":96,"threads_started":1}
I20260809 00:33:09.474139 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.462984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34856 (local address 127.21.41.252:43915)
0809 00:33:09.463381 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.463383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.463649 (+ 266us) server_negotiation.cc:407] Connection header received
0809 00:33:09.463673 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.463674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.463688 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.463743 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.465717 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.467456 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.468396 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.468702 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.468765 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.470768 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.470769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.470770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.470777 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.472547 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.473008 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.473009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.473009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.473074 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.473268 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.473269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.473269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.473398 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.473507 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.473856 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.473944 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":54,"threads_started":1}
I20260809 00:33:09.474478 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.462593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34814 (local address 127.21.41.253:35627)
0809 00:33:09.464462 (+ 1869us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.464464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.464472 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:33:09.464489 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.464489 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.464498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.464548 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.467151 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.467864 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.468896 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.469097 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.469170 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.470971 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.470972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.470972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.470979 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.473063 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.473492 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.473492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.473493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.473542 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.473736 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.473738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.473738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.473873 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.473963 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.474256 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.474366 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1848,"thread_start_us":59,"threads_started":1}
I20260809 00:33:09.475762 27789 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:33:09.477324 27789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:33:09.484828 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.481090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:42291 (local address 127.0.0.1:54628)
0809 00:33:09.481337 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.481364 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.481407 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.481674 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.481675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.481677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:09.481737 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.481738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.482862 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.482864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.483701 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.483703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.484644 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.484645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.484706 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.484710 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.484741 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.484775 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229}
I20260809 00:33:09.485389 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.481282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54628 (local address 127.21.41.195:42291)
0809 00:33:09.481490 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.481495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.481505 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:09.481548 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.481550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.481569 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.481822 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:09.481881 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.482777 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.483796 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.484542 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.485134 (+ 592us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.485234 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.485260 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.485295 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":96,"threads_started":1}
I20260809 00:33:09.485539 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.481280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44230 (local address 127.21.41.194:32937)
0809 00:33:09.481458 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.481464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.481542 (+ 78us) server_negotiation.cc:407] Connection header received
0809 00:33:09.481633 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.481636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.481651 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.481723 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:09.481920 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.483389 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.484459 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.485153 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.485229 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.485301 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.485395 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.485442 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260809 00:33:09.485759 29060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.481121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38675 (local address 127.0.0.1:36702)
0809 00:33:09.481802 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.481812 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.481885 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.482147 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.482148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.482150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:09.482185 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.482186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.483476 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.483477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.484559 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.484562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.485533 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.485535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.485603 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.485607 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.485662 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.485699 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":95,"threads_started":1}
I20260809 00:33:09.486019 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.481505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.21.41.193:38675)
0809 00:33:09.481746 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.481750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.481858 (+ 108us) server_negotiation.cc:407] Connection header received
0809 00:33:09.482027 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.482031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.482052 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.482136 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:09.482260 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.483383 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.484644 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.485367 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.485433 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.485497 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.485876 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.485927 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":79,"threads_started":1}
I20260809 00:33:09.486147 28972 tablet_service.cc:1467] Processing CreateTablet for tablet d06aacf524d44962bad85c25c9bad09c (DEFAULT_TABLE table=test-table [id=62f6bcef069740d983d36fdba5514771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:09.486549 28972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d06aacf524d44962bad85c25c9bad09c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:09.486634 28697 tablet_service.cc:1467] Processing CreateTablet for tablet d06aacf524d44962bad85c25c9bad09c (DEFAULT_TABLE table=test-table [id=62f6bcef069740d983d36fdba5514771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:09.486893 28697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d06aacf524d44962bad85c25c9bad09c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:09.489634 29064 tablet_bootstrap.cc:492] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: Bootstrap starting.
I20260809 00:33:09.489784 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.481090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:32937 (local address 127.0.0.1:44230)
0809 00:33:09.481488 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.481498 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.481557 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.481793 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.481794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.481796 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:09.481842 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.481844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.483492 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.483493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.484367 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.484370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.485248 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.485249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.485323 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.485326 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.485385 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.485427 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":23,"thread_start_us":111,"threads_started":1}
I20260809 00:33:09.490274 29065 tablet_bootstrap.cc:492] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: Bootstrap starting.
I20260809 00:33:09.490592 29064 tablet_bootstrap.cc:654] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:09.490574 28835 tablet_service.cc:1467] Processing CreateTablet for tablet d06aacf524d44962bad85c25c9bad09c (DEFAULT_TABLE table=test-table [id=62f6bcef069740d983d36fdba5514771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:09.490955 29065 tablet_bootstrap.cc:654] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:09.490958 28835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d06aacf524d44962bad85c25c9bad09c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:09.491017 29064 log.cc:824] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:09.491282 29065 log.cc:824] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:09.491868 29064 tablet_bootstrap.cc:492] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: No bootstrap required, opened a new log
I20260809 00:33:09.491952 29064 ts_tablet_manager.cc:1397] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:33:09.492069 29065 tablet_bootstrap.cc:492] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: No bootstrap required, opened a new log
I20260809 00:33:09.492141 29065 ts_tablet_manager.cc:1397] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:33:09.493863 29064 raft_consensus.cc:348] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:09.494031 29064 raft_consensus.cc:374] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:09.494076 29064 raft_consensus.cc:729] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ea8b2562624f53a4c81f0d3ee0be50, State: Initialized, Role: FOLLOWER
I20260809 00:33:09.494196 29064 consensus_queue.cc:260] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:09.494498 29064 ts_tablet_manager.cc:1428] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:33:09.494688 29065 raft_consensus.cc:348] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:09.494855 29039 heartbeater.cc:503] Master 127.21.41.254:46427 was elected leader, sending a full tablet report...
I20260809 00:33:09.494856 29065 raft_consensus.cc:374] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:09.494870 29069 tablet_bootstrap.cc:492] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: Bootstrap starting.
I20260809 00:33:09.494899 29065 raft_consensus.cc:729] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 660fd2a7b01b4e15a201d53f7fe61e58, State: Initialized, Role: FOLLOWER
I20260809 00:33:09.495023 29065 consensus_queue.cc:260] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:09.495273 29065 ts_tablet_manager.cc:1428] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 00:33:09.495677 28764 heartbeater.cc:503] Master 127.21.41.254:46427 was elected leader, sending a full tablet report...
I20260809 00:33:09.496692 29069 tablet_bootstrap.cc:654] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:09.497021 29069 log.cc:824] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:09.497650 29069 tablet_bootstrap.cc:492] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: No bootstrap required, opened a new log
I20260809 00:33:09.497702 29069 ts_tablet_manager.cc:1397] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:33:09.499279 29069 raft_consensus.cc:348] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:09.499395 29069 raft_consensus.cc:374] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:09.499416 29069 raft_consensus.cc:729] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf1f7a8afa5a42a7853b7ad1c2ee7855, State: Initialized, Role: FOLLOWER
I20260809 00:33:09.499490 29069 consensus_queue.cc:260] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:09.499662 29069 ts_tablet_manager.cc:1428] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:33:09.499734 28902 heartbeater.cc:503] Master 127.21.41.254:46427 was elected leader, sending a full tablet report...
I20260809 00:33:09.505934 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.503185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.21.41.252:43915)
0809 00:33:09.503251 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.503253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.503276 (+ 23us) server_negotiation.cc:407] Connection header received
0809 00:33:09.503298 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.503299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.503317 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.503362 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:09.503479 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.504197 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.505133 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.505601 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.505658 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.505708 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.505850 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.505883 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:33:09.505937 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.503112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:43915 (local address 127.0.0.1:34866)
0809 00:33:09.503179 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.503193 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.503246 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.503389 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.503390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.503391 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:09.503437 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.503439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.504252 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.504253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.505024 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.505027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.505703 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.505705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.505782 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.505786 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.505839 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.505884 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 00:33:09.506289 28594 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260809 00:33:09.506785 28623 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4914f52a2dc4d7eaf6c0d04db25f582. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.506879 28623 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: This master's current role is: LEARNER
W20260809 00:33:09.570281 28904 tablet.cc:2369] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:33:09.676749 28766 tablet.cc:2369] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:33:09.702598 29041 tablet.cc:2369] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:33:09.711457 29068 raft_consensus.cc:482] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:33:09.711566 29068 raft_consensus.cc:504] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:09.711999 29068 leader_election.cc:288] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf1f7a8afa5a42a7853b7ad1c2ee7855 (127.21.41.194:32937), 660fd2a7b01b4e15a201d53f7fe61e58 (127.21.41.193:38675)
I20260809 00:33:09.714845 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.712251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:43657 (local address 127.21.41.194:32937)
0809 00:33:09.712318 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.712320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.712329 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:09.712356 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.712356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.712366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.712436 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:09.712620 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.713100 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.714088 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.714470 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.714553 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.714563 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.714734 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.714768 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 00:33:09.714869 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.712168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:32937 (local address 127.21.41.195:43657)
0809 00:33:09.712231 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.712244 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.712286 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.712455 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.712456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.712457 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:09.712532 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.712534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.713184 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.713185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.714000 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.714002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.714563 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.714564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.714640 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.714643 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.714716 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.714754 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":38}
I20260809 00:33:09.714874 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.712273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:37569 (local address 127.21.41.193:38675)
0809 00:33:09.712316 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.712318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.712329 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:33:09.712401 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.712402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.712412 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.712483 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:09.712636 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.713104 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.714072 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.714492 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.714579 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.714587 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.714758 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.714794 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:33:09.714975 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.712173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38675 (local address 127.21.41.195:37569)
0809 00:33:09.712289 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:09.712301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:09.712346 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:09.712485 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:09.712486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:09.712487 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:09.712562 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.712563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.713191 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.713192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.713976 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:09.713978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.714579 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.714580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:09.714669 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.714671 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:33:09.714740 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 00:33:09.714779 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":124}
I20260809 00:33:09.715310 28855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d06aacf524d44962bad85c25c9bad09c" candidate_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" is_pre_election: true
I20260809 00:33:09.715435 28855 raft_consensus.cc:2454] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ea8b2562624f53a4c81f0d3ee0be50 in term 0.
I20260809 00:33:09.715627 28717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d06aacf524d44962bad85c25c9bad09c" candidate_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" is_pre_election: true
I20260809 00:33:09.715677 28926 leader_election.cc:302] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ea8b2562624f53a4c81f0d3ee0be50, cf1f7a8afa5a42a7853b7ad1c2ee7855; no voters:
I20260809 00:33:09.715778 28717 raft_consensus.cc:2454] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ea8b2562624f53a4c81f0d3ee0be50 in term 0.
I20260809 00:33:09.715801 29068 raft_consensus.cc:2781] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:33:09.715875 29068 raft_consensus.cc:482] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:33:09.715911 29068 raft_consensus.cc:3037] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:09.716619 29068 raft_consensus.cc:504] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:09.716766 29068 leader_election.cc:288] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [CANDIDATE]: Term 1 election: Requested vote from peers cf1f7a8afa5a42a7853b7ad1c2ee7855 (127.21.41.194:32937), 660fd2a7b01b4e15a201d53f7fe61e58 (127.21.41.193:38675)
I20260809 00:33:09.716928 28855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d06aacf524d44962bad85c25c9bad09c" candidate_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855"
I20260809 00:33:09.717000 28855 raft_consensus.cc:3037] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:09.717731 28855 raft_consensus.cc:2454] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ea8b2562624f53a4c81f0d3ee0be50 in term 1.
I20260809 00:33:09.717892 28926 leader_election.cc:302] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ea8b2562624f53a4c81f0d3ee0be50, cf1f7a8afa5a42a7853b7ad1c2ee7855; no voters:
I20260809 00:33:09.717866 28717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d06aacf524d44962bad85c25c9bad09c" candidate_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "660fd2a7b01b4e15a201d53f7fe61e58"
I20260809 00:33:09.717945 28717 raft_consensus.cc:3037] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:09.717995 29068 raft_consensus.cc:2781] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:09.718174 29068 raft_consensus.cc:686] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 1 LEADER]: Becoming Leader. State: Replica: 53ea8b2562624f53a4c81f0d3ee0be50, State: Running, Role: LEADER
I20260809 00:33:09.718276 29068 consensus_queue.cc:237] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:09.718817 28717 raft_consensus.cc:2454] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ea8b2562624f53a4c81f0d3ee0be50 in term 1.
I20260809 00:33:09.719178 27789 catalog_manager.cc:5495] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ea8b2562624f53a4c81f0d3ee0be50 (127.21.41.195). New cstate: current_term: 1 leader_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } health_report { overall_health: UNKNOWN } } }
I20260809 00:33:09.719869 28548 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: NON_VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: true } } }
I20260809 00:33:09.719971 28548 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:09.720436 28547 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582: attempting to promote NON_VOTER a02a760b5dd940f4adcd6227da071be3 to VOTER
I20260809 00:33:09.720573 28547 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } }
I20260809 00:33:09.720975 28594 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 LEARNER]: Refusing update from remote peer d4914f52a2dc4d7eaf6c0d04db25f582: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 00:33:09.721019 28391 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Refusing update from remote peer d4914f52a2dc4d7eaf6c0d04db25f582: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 00:33:09.721177 29079 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260809 00:33:09.721257 28547 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260809 00:33:09.721781 28547 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a02a760b5dd940f4adcd6227da071be3 (127.21.41.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:09.721952 28391 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a02a760b5dd940f4adcd6227da071be3 (127.21.41.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:09.722636 28547 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:09.722718 28547 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:09.722792 28547 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:09.722842 28547 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:09.723031 29082 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:09.723145 29082 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:33:09.723162 29081 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:09.723251 29081 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:09.723291 28594 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a02a760b5dd940f4adcd6227da071be3 (127.21.41.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:09.724076 28623 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:09.724160 28623 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:33:09.768406 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.766268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54638 (local address 127.21.41.195:42291)
0809 00:33:09.766320 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.766322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.766329 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:33:09.766404 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.766405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.766424 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.766485 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:33:09.766624 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.767098 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.767786 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.767964 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.768014 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.768144 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:33:09.768243 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:33:09.768302 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.768331 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.768348 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:33:09.770435 28855 raft_consensus.cc:1261] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 1 FOLLOWER]: Refusing update from remote peer 53ea8b2562624f53a4c81f0d3ee0be50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:09.770411 28717 raft_consensus.cc:1261] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58 [term 1 FOLLOWER]: Refusing update from remote peer 53ea8b2562624f53a4c81f0d3ee0be50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:09.770816 29077 consensus_queue.cc:1035] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:09.770936 29068 consensus_queue.cc:1035] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:09.773507 29084 mvcc.cc:205] Tried to move back new op lower bound from 7316420975696531456 to 7316420975486832640. Current Snapshot: MvccSnapshot[applied={T|T < 7316420975696531456}]
I20260809 00:33:09.774428 29086 mvcc.cc:205] Tried to move back new op lower bound from 7316420975696531456 to 7316420975486832640. Current Snapshot: MvccSnapshot[applied={T|T < 7316420975696531456 or (T in {7316420975696531456})}]
I20260809 00:33:09.776037 29085 mvcc.cc:205] Tried to move back new op lower bound from 7316420975696531456 to 7316420975486832640. Current Snapshot: MvccSnapshot[applied={T|T < 7316420975696531456}]
I20260809 00:33:09.777282 27789 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:09.777392 27789 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:09.778872 27789 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4914f52a2dc4d7eaf6c0d04db25f582: Sending DeleteTablet for 3 replicas of tablet d06aacf524d44962bad85c25c9bad09c
I20260809 00:33:09.779213 28972 tablet_service.cc:1514] Processing DeleteTablet for tablet d06aacf524d44962bad85c25c9bad09c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:09 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:54628
I20260809 00:33:09.779212 28835 tablet_service.cc:1514] Processing DeleteTablet for tablet d06aacf524d44962bad85c25c9bad09c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:09 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:44230
I20260809 00:33:09.779488 28697 tablet_service.cc:1514] Processing DeleteTablet for tablet d06aacf524d44962bad85c25c9bad09c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:09 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:36702
I20260809 00:33:09.779764 29096 tablet_replica.cc:330] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: stopping tablet replica
I20260809 00:33:09.779868 29096 raft_consensus.cc:2229] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:09.779976 29096 raft_consensus.cc:2258] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:09.780247 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 27757
I20260809 00:33:09.781595 29096 ts_tablet_manager.cc:1905] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:09.782472 29094 tablet_replica.cc:330] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: stopping tablet replica
I20260809 00:33:09.782606 29094 raft_consensus.cc:2229] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:09.782729 29094 raft_consensus.cc:2258] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:09.782900 29095 tablet_replica.cc:330] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: stopping tablet replica
I20260809 00:33:09.783012 29095 raft_consensus.cc:2229] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:33:09.783123 29095 raft_consensus.cc:2258] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:09.783346 29094 ts_tablet_manager.cc:1905] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:09.783545 29096 ts_tablet_manager.cc:1918] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:09.783618 29096 log.cc:1196] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-0/wal/wals/d06aacf524d44962bad85c25c9bad09c
I20260809 00:33:09.783885 29096 ts_tablet_manager.cc:1939] T d06aacf524d44962bad85c25c9bad09c P 660fd2a7b01b4e15a201d53f7fe61e58: Deleting consensus metadata
I20260809 00:33:09.785066 29094 ts_tablet_manager.cc:1918] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:09.785145 29094 log.cc:1196] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/d06aacf524d44962bad85c25c9bad09c
I20260809 00:33:09.785420 29094 ts_tablet_manager.cc:1939] T d06aacf524d44962bad85c25c9bad09c P cf1f7a8afa5a42a7853b7ad1c2ee7855: Deleting consensus metadata
I20260809 00:33:09.783550 29095 ts_tablet_manager.cc:1905] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:33:09.786829 28650 connection.cc:535] server connection from 127.0.0.1:36702 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 00:33:09.787063 28787 connection.cc:535] server connection from 127.0.0.1:44230 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 00:33:09.788166 29095 ts_tablet_manager.cc:1918] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:09.788326 29095 log.cc:1196] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/d06aacf524d44962bad85c25c9bad09c
I20260809 00:33:09.788700 29095 ts_tablet_manager.cc:1939] T d06aacf524d44962bad85c25c9bad09c P 53ea8b2562624f53a4c81f0d3ee0be50: Deleting consensus metadata
W20260809 00:33:09.789067 28928 connection.cc:441] server connection from 127.0.0.1:54628 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54628 (request call id 1) could send its response
I20260809 00:33:09.796726 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.779867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34868 (local address 127.21.41.252:43915)
0809 00:33:09.779909 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.779912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.788243 (+ 8331us) server_negotiation.cc:407] Connection header received
0809 00:33:09.788360 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.788361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.788373 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.788403 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.789331 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.790106 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.791218 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.791519 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.791560 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.793853 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.793854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.793855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.793863 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.795588 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.795931 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.795932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.795933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.795985 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.796195 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.796196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.796197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.796317 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.796422 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.796533 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.796606 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:33:09.798184 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.789217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34826 (local address 127.21.41.253:35627)
0809 00:33:09.789260 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.789261 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.789530 (+ 269us) server_negotiation.cc:407] Connection header received
0809 00:33:09.789657 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.789669 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.789682 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.789715 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.790570 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.791262 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.792256 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.792566 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.792612 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.794251 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.794252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.794253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.794259 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.797199 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.797552 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.797554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.797554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.797608 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.797739 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.797740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.797740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.797863 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.797958 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.798019 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.798091 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:33:09.799417 28767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:09.790249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34874 (local address 127.21.41.252:43915)
0809 00:33:09.790301 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:09.790303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:09.790309 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:33:09.790329 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:09.790329 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:09.790370 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:09.790420 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:09.791145 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.791856 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.792888 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:09.793181 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:09.793232 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:09.794718 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:09.794719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:09.794720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:09.794726 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:09.796484 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.798626 (+ 2142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.798628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.798628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.798688 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:09.798945 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:09.798946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:09.798946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:09.799045 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:09.799147 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:09.799241 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 00:33:09.799318 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:33:10.775159 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:10.774788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.194:41113)
0809 00:33:10.774921 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:10.775015 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
W20260809 00:33:10.775569 28902 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:46427 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
I20260809 00:33:10.776209 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:10.775926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.195:50331)
0809 00:33:10.776056 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:10.776159 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":50,"threads_started":1}
W20260809 00:33:10.776438 29039 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:46427 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
I20260809 00:33:10.777972 29107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:10.777621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.193:59149)
0809 00:33:10.777853 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:10.777926 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":150,"threads_started":1}
W20260809 00:33:10.778168 28764 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:46427 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
I20260809 00:33:11.279948 29108 raft_consensus.cc:482] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4914f52a2dc4d7eaf6c0d04db25f582)
I20260809 00:33:11.280110 29108 raft_consensus.cc:504] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } }
I20260809 00:33:11.280478 29108 leader_election.cc:288] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427), b71d8be91b3648bd82ab63e06504fe5c (127.21.41.253:35627)
I20260809 00:33:11.280478 29109 raft_consensus.cc:482] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4914f52a2dc4d7eaf6c0d04db25f582)
I20260809 00:33:11.280611 29109 raft_consensus.cc:504] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } }
I20260809 00:33:11.280859 29109 leader_election.cc:288] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427), a02a760b5dd940f4adcd6227da071be3 (127.21.41.252:43915)
I20260809 00:33:11.280867 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.280610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59822)
0809 00:33:11.280753 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.280815 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":52,"threads_started":1}
I20260809 00:33:11.281217 29113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.280940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59832)
0809 00:33:11.281058 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.281126 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
W20260809 00:33:11.281315 28569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
W20260809 00:33:11.281380 28374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
I20260809 00:33:11.281756 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.281657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59844)
0809 00:33:11.281701 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.281727 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 00:33:11.281839 28569 leader_election.cc:334] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427): Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
I20260809 00:33:11.281980 29114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.281744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59860)
0809 00:33:11.281867 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.281946 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 00:33:11.282068 28374 leader_election.cc:334] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427): Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
I20260809 00:33:11.285035 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.280638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35627 (local address 127.0.0.1:34842)
0809 00:33:11.280871 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.280884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:11.280953 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:11.281307 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:11.281309 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:11.281312 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:11.281388 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:11.281390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:11.282846 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:11.282847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:11.284011 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:11.284014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:11.284737 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:11.284738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:11.284800 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:11.284805 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:33:11.284865 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 00:33:11.284913 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":47,"threads_started":1}
I20260809 00:33:11.285039 29115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.281216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.21.41.252:43915)
0809 00:33:11.281508 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:11.281510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:11.281518 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:33:11.281539 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:11.281540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:11.281550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:11.281778 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:11.281814 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:11.282588 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:11.283975 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:11.284640 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:11.284730 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:11.284741 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:11.284885 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 00:33:11.284931 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":55,"threads_started":1}
I20260809 00:33:11.285032 29113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.281111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:43915 (local address 127.0.0.1:34888)
0809 00:33:11.281294 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.281305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:11.281356 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:11.281664 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:11.281665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:11.281666 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:11.281703 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:11.281704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:11.282667 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:11.282668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:11.283864 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:11.283867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:11.284738 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:11.284739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:11.284808 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:11.284813 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:33:11.284864 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:33:11.284916 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":6,"thread_start_us":45,"threads_started":1}
I20260809 00:33:11.285030 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.280736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34842 (local address 127.21.41.253:35627)
0809 00:33:11.281000 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:11.281002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:11.281011 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:11.281042 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:11.281043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:11.281074 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:11.281465 (+ 391us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:11.281510 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:11.282771 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:11.284115 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:11.284651 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:11.284727 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:11.284740 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:11.284885 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 00:33:11.284931 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":90,"threads_started":1}
I20260809 00:33:11.285329 28594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b71d8be91b3648bd82ab63e06504fe5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a02a760b5dd940f4adcd6227da071be3" is_pre_election: true
I20260809 00:33:11.285341 28391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a02a760b5dd940f4adcd6227da071be3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b71d8be91b3648bd82ab63e06504fe5c" is_pre_election: true
I20260809 00:33:11.285426 28594 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b71d8be91b3648bd82ab63e06504fe5c in term 1.
I20260809 00:33:11.285451 28391 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a02a760b5dd940f4adcd6227da071be3 in term 1.
I20260809 00:33:11.285679 28570 leader_election.cc:302] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a02a760b5dd940f4adcd6227da071be3, b71d8be91b3648bd82ab63e06504fe5c; no voters: d4914f52a2dc4d7eaf6c0d04db25f582
I20260809 00:33:11.285682 28374 leader_election.cc:302] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a02a760b5dd940f4adcd6227da071be3, b71d8be91b3648bd82ab63e06504fe5c; no voters: d4914f52a2dc4d7eaf6c0d04db25f582
I20260809 00:33:11.285804 29108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 00:33:11.285825 29109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 00:33:11.285851 29108 raft_consensus.cc:482] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d4914f52a2dc4d7eaf6c0d04db25f582)
I20260809 00:33:11.285857 29109 raft_consensus.cc:482] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Starting leader election (detected failure of leader d4914f52a2dc4d7eaf6c0d04db25f582)
I20260809 00:33:11.285884 29109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:33:11.285884 29108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:33:11.286795 29109 raft_consensus.cc:504] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } }
I20260809 00:33:11.286795 29108 raft_consensus.cc:504] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } }
I20260809 00:33:11.286921 29109 leader_election.cc:288] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [CANDIDATE]: Term 2 election: Requested vote from peers d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427), a02a760b5dd940f4adcd6227da071be3 (127.21.41.252:43915)
I20260809 00:33:11.286928 29108 leader_election.cc:288] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 2 election: Requested vote from peers d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427), b71d8be91b3648bd82ab63e06504fe5c (127.21.41.253:35627)
I20260809 00:33:11.287174 29113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.287055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59874)
0809 00:33:11.287103 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.287148 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 00:33:11.287220 28391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a02a760b5dd940f4adcd6227da071be3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b71d8be91b3648bd82ab63e06504fe5c"
I20260809 00:33:11.287199 28594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b71d8be91b3648bd82ab63e06504fe5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a02a760b5dd940f4adcd6227da071be3"
I20260809 00:33:11.287293 28391 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a02a760b5dd940f4adcd6227da071be3 in current term 2: Already voted for candidate b71d8be91b3648bd82ab63e06504fe5c in this term.
I20260809 00:33:11.287303 28594 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b71d8be91b3648bd82ab63e06504fe5c in current term 2: Already voted for candidate a02a760b5dd940f4adcd6227da071be3 in this term.
I20260809 00:33:11.287207 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.287055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59876)
0809 00:33:11.287154 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.287183 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 00:33:11.287539 29113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.287386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59884)
0809 00:33:11.287484 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.287513 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260809 00:33:11.287680 28374 leader_election.cc:334] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427): Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
I20260809 00:33:11.287748 28374 leader_election.cc:302] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b71d8be91b3648bd82ab63e06504fe5c; no voters: a02a760b5dd940f4adcd6227da071be3, d4914f52a2dc4d7eaf6c0d04db25f582
I20260809 00:33:11.287810 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.287708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:59898)
0809 00:33:11.287760 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.287789 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 00:33:11.287813 29109 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260809 00:33:11.287873 28569 leader_election.cc:334] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427): Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
I20260809 00:33:11.287911 28569 leader_election.cc:302] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a02a760b5dd940f4adcd6227da071be3; no voters: b71d8be91b3648bd82ab63e06504fe5c, d4914f52a2dc4d7eaf6c0d04db25f582
I20260809 00:33:11.287988 29108 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 00:33:11.776206 29118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.775943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.194:58763)
0809 00:33:11.776097 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.776143 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":110,"threads_started":1}
I20260809 00:33:11.776877 29119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.776699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.195:35841)
0809 00:33:11.776796 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.776842 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":47,"threads_started":1}
I20260809 00:33:11.778678 29120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:11.778480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.193:57463)
0809 00:33:11.778619 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:11.778645 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
I20260809 00:33:12.781028 29121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.780547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.194:40451)
0809 00:33:12.780916 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.780962 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":68,"threads_started":1}
I20260809 00:33:12.781679 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.780858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.195:34461)
0809 00:33:12.781590 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.781634 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":55,"threads_started":1}
I20260809 00:33:12.781962 29123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.781155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.193:58359)
0809 00:33:12.781876 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.781912 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":48,"threads_started":1}
I20260809 00:33:12.795248 29124 raft_consensus.cc:482] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:33:12.795341 29124 raft_consensus.cc:504] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } }
I20260809 00:33:12.795504 29124 leader_election.cc:288] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427), b71d8be91b3648bd82ab63e06504fe5c (127.21.41.253:35627)
I20260809 00:33:12.795795 28391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a02a760b5dd940f4adcd6227da071be3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b71d8be91b3648bd82ab63e06504fe5c" is_pre_election: true
I20260809 00:33:12.795930 28391 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a02a760b5dd940f4adcd6227da071be3 in term 2.
I20260809 00:33:12.795959 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.795755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:60092)
0809 00:33:12.795870 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.795908 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
I20260809 00:33:12.796096 28570 leader_election.cc:302] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a02a760b5dd940f4adcd6227da071be3, b71d8be91b3648bd82ab63e06504fe5c; no voters:
I20260809 00:33:12.796226 29124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 00:33:12.796299 29124 raft_consensus.cc:482] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:33:12.796330 29124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 00:33:12.796398 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.796280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:60108)
0809 00:33:12.796329 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.796356 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 00:33:12.796506 28569 leader_election.cc:334] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427): Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
I20260809 00:33:12.797119 29124 raft_consensus.cc:504] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } }
I20260809 00:33:12.797261 29124 leader_election.cc:288] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 3 election: Requested vote from peers d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427), b71d8be91b3648bd82ab63e06504fe5c (127.21.41.253:35627)
I20260809 00:33:12.797389 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.797318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:60114)
0809 00:33:12.797347 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.797368 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 00:33:12.797415 28391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a02a760b5dd940f4adcd6227da071be3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b71d8be91b3648bd82ab63e06504fe5c"
I20260809 00:33:12.797492 28391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 2 FOLLOWER]: Advancing to term 3
I20260809 00:33:12.797703 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.797617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:60118)
0809 00:33:12.797661 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.797683 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:33:12.797789 28569 leader_election.cc:334] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427): Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
I20260809 00:33:12.798223 28391 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a02a760b5dd940f4adcd6227da071be3 in term 3.
I20260809 00:33:12.798444 28570 leader_election.cc:302] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a02a760b5dd940f4adcd6227da071be3, b71d8be91b3648bd82ab63e06504fe5c; no voters: d4914f52a2dc4d7eaf6c0d04db25f582
I20260809 00:33:12.798544 29124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 00:33:12.798647 29124 raft_consensus.cc:686] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [term 3 LEADER]: Becoming Leader. State: Replica: a02a760b5dd940f4adcd6227da071be3, State: Running, Role: LEADER
I20260809 00:33:12.798727 29124 consensus_queue.cc:237] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } }
I20260809 00:33:12.799108 29127 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a02a760b5dd940f4adcd6227da071be3. Latest consensus state: current_term: 3 leader_uuid: "a02a760b5dd940f4adcd6227da071be3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:12.799211 29127 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:12.799407 29129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:12.799783 29129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:12.799834 29129 catalog_manager.cc:1259] Loaded cluster ID: 5495401561c54a1887cfc553204f7b88
I20260809 00:33:12.799851 29129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:33:12.799993 29129 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:12.800045 29129 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: Loaded TSK: 0
I20260809 00:33:12.800145 29129 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:33:12.806125 28584 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:33:12.807685 28584 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:33:12.808583 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.808477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:60144)
0809 00:33:12.808526 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.808559 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:33:12.808583 28391 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [term 3 FOLLOWER]: Refusing update from remote peer a02a760b5dd940f4adcd6227da071be3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 00:33:12.808813 29127 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 00:33:12.808930 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.808837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.0.0.1:60154)
0809 00:33:12.808881 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.808912 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 00:33:12.809062 28569 consensus_peers.cc:487] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 -> Peer d4914f52a2dc4d7eaf6c0d04db25f582 (127.21.41.254:46427): Couldn't send request to peer d4914f52a2dc4d7eaf6c0d04db25f582. Status: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 00:33:12.809425 29131 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a02a760b5dd940f4adcd6227da071be3. Latest consensus state: current_term: 3 leader_uuid: "a02a760b5dd940f4adcd6227da071be3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:12.809515 29131 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:33:12.810010 29127 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a02a760b5dd940f4adcd6227da071be3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:12.810099 29127 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:12.810062 29131 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "a02a760b5dd940f4adcd6227da071be3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4914f52a2dc4d7eaf6c0d04db25f582" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46427 } } peers { permanent_uuid: "b71d8be91b3648bd82ab63e06504fe5c" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35627 } attrs { promote: false } } peers { permanent_uuid: "a02a760b5dd940f4adcd6227da071be3" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 43915 } attrs { promote: false } } }
I20260809 00:33:12.810145 29131 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71d8be91b3648bd82ab63e06504fe5c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:33:12.810200 29132 mvcc.cc:205] Tried to move back new op lower bound from 7316420988142288896 to 7316420988104409088. Current Snapshot: MvccSnapshot[applied={T|T < 7316420988142288896}]
I20260809 00:33:12.815663 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.812008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38675 (local address 127.0.0.1:36704)
0809 00:33:12.812742 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.812751 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:12.812794 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:12.812952 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:12.812953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:12.812955 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:12.813003 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:12.813004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.813824 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.813825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:12.814707 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:12.814709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.815408 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.815409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:12.815469 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:12.815473 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:33:12.815537 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 00:33:12.815592 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260809 00:33:12.815703 29140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.812345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.21.41.193:38675)
0809 00:33:12.812451 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:12.812453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:12.812836 (+ 383us) server_negotiation.cc:407] Connection header received
0809 00:33:12.812865 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:12.812866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:12.812878 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:12.812939 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:12.813067 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.813729 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.814805 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.815325 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.815396 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:12.815407 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:12.815551 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 00:33:12.815607 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":59,"threads_started":1}
I20260809 00:33:12.815920 28697 tablet_service.cc:1467] Processing CreateTablet for tablet 558ca78be93d45f1b60b66a17205693f (DEFAULT_TABLE table=test-table [id=2291298febd847b097caefda8a88f36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:12.815950 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.811938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:42291 (local address 127.0.0.1:54642)
0809 00:33:12.812140 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.812151 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:12.812216 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:12.812515 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:12.812516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:12.812518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:12.812588 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:12.812590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.813653 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.813654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:12.814955 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:12.814957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.815703 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.815704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:12.815783 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:12.815788 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:33:12.815819 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 00:33:12.815865 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":69}
I20260809 00:33:12.816030 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.812201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44238 (local address 127.21.41.194:32937)
0809 00:33:12.812359 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:12.812361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:12.812839 (+ 478us) server_negotiation.cc:407] Connection header received
0809 00:33:12.812867 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:12.812868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:12.812878 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:12.812963 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:12.813113 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.813583 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.815057 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.815486 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.815545 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:12.815566 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:12.815914 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 00:33:12.815963 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260809 00:33:12.816059 28697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558ca78be93d45f1b60b66a17205693f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:12.816075 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.812008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:32937 (local address 127.0.0.1:44238)
0809 00:33:12.812594 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.812604 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:12.812651 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:12.813007 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:12.813008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:12.813009 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:12.813044 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:12.813046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.813675 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.813676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:12.814955 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:12.814957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.815596 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.815596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:12.815654 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:12.815657 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:12.815703 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 00:33:12.815851 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":37,"thread_start_us":87,"threads_started":1}
I20260809 00:33:12.816268 28835 tablet_service.cc:1467] Processing CreateTablet for tablet 558ca78be93d45f1b60b66a17205693f (DEFAULT_TABLE table=test-table [id=2291298febd847b097caefda8a88f36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:12.816409 28835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558ca78be93d45f1b60b66a17205693f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:12.816808 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.812044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.21.41.195:42291)
0809 00:33:12.812377 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:12.812380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:12.812388 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:33:12.812416 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:12.812416 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:12.812429 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:12.812498 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:12.812761 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.813557 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.815039 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.815609 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.816642 (+ 1033us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:12.816657 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:12.816677 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 00:33:12.816730 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":63,"threads_started":1}
I20260809 00:33:12.817632 29141 tablet_bootstrap.cc:492] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58: Bootstrap starting.
I20260809 00:33:12.817701 28972 tablet_service.cc:1467] Processing CreateTablet for tablet 558ca78be93d45f1b60b66a17205693f (DEFAULT_TABLE table=test-table [id=2291298febd847b097caefda8a88f36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:12.817819 28972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558ca78be93d45f1b60b66a17205693f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:12.818040 29141 tablet_bootstrap.cc:654] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:12.818740 29141 tablet_bootstrap.cc:492] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58: No bootstrap required, opened a new log
I20260809 00:33:12.818804 29141 ts_tablet_manager.cc:1397] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:12.818948 29141 raft_consensus.cc:348] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:12.819016 29141 raft_consensus.cc:374] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:12.819036 29141 raft_consensus.cc:729] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 660fd2a7b01b4e15a201d53f7fe61e58, State: Initialized, Role: FOLLOWER
I20260809 00:33:12.819103 29141 consensus_queue.cc:260] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:12.819314 29141 ts_tablet_manager.cc:1428] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:12.819386 29145 tablet_bootstrap.cc:492] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50: Bootstrap starting.
I20260809 00:33:12.819538 29142 tablet_bootstrap.cc:492] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855: Bootstrap starting.
I20260809 00:33:12.819648 29123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.819487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.193:41699)
0809 00:33:12.819594 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.819621 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 00:33:12.819778 29145 tablet_bootstrap.cc:654] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:12.819917 29142 tablet_bootstrap.cc:654] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:12.820173 28763 heartbeater.cc:503] Master 127.21.41.252:43915 was elected leader, sending a full tablet report...
I20260809 00:33:12.820324 29123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.820228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.193:45575)
0809 00:33:12.820267 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.820299 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 00:33:12.820449 29145 tablet_bootstrap.cc:492] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50: No bootstrap required, opened a new log
I20260809 00:33:12.820528 29145 ts_tablet_manager.cc:1397] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:12.820557 29142 tablet_bootstrap.cc:492] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855: No bootstrap required, opened a new log
I20260809 00:33:12.820595 29142 ts_tablet_manager.cc:1397] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:12.820643 29145 raft_consensus.cc:348] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:12.820729 29145 raft_consensus.cc:374] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:12.820758 29145 raft_consensus.cc:729] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ea8b2562624f53a4c81f0d3ee0be50, State: Initialized, Role: FOLLOWER
I20260809 00:33:12.820729 29142 raft_consensus.cc:348] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:12.820796 29142 raft_consensus.cc:374] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:12.820825 29142 raft_consensus.cc:729] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf1f7a8afa5a42a7853b7ad1c2ee7855, State: Initialized, Role: FOLLOWER
I20260809 00:33:12.820811 29145 consensus_queue.cc:260] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:12.820892 29142 consensus_queue.cc:260] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:12.820984 29145 ts_tablet_manager.cc:1428] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:12.821077 29142 ts_tablet_manager.cc:1428] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:12.821491 29121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.821396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.194:57757)
0809 00:33:12.821430 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.821462 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 00:33:12.821849 29121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.821764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.194:49915)
0809 00:33:12.821795 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.821823 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 00:33:12.821934 28901 heartbeater.cc:503] Master 127.21.41.252:43915 was elected leader, sending a full tablet report...
I20260809 00:33:12.822042 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.821936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.195:47309)
0809 00:33:12.821978 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.822010 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 00:33:12.822039 29038 heartbeater.cc:503] Master 127.21.41.252:43915 was elected leader, sending a full tablet report...
I20260809 00:33:12.869306 29148 raft_consensus.cc:482] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:33:12.869453 29148 raft_consensus.cc:504] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:12.869678 29148 leader_election.cc:288] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf1f7a8afa5a42a7853b7ad1c2ee7855 (127.21.41.194:32937), 660fd2a7b01b4e15a201d53f7fe61e58 (127.21.41.193:38675)
I20260809 00:33:12.869972 28717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558ca78be93d45f1b60b66a17205693f" candidate_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" is_pre_election: true
I20260809 00:33:12.869963 28855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558ca78be93d45f1b60b66a17205693f" candidate_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" is_pre_election: true
I20260809 00:33:12.870095 28855 raft_consensus.cc:2454] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ea8b2562624f53a4c81f0d3ee0be50 in term 0.
I20260809 00:33:12.870095 28717 raft_consensus.cc:2454] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ea8b2562624f53a4c81f0d3ee0be50 in term 0.
I20260809 00:33:12.870293 28927 leader_election.cc:302] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ea8b2562624f53a4c81f0d3ee0be50, 660fd2a7b01b4e15a201d53f7fe61e58; no voters:
I20260809 00:33:12.870465 29148 raft_consensus.cc:2781] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:33:12.870512 29148 raft_consensus.cc:482] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:33:12.870538 29148 raft_consensus.cc:3037] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:12.871335 29148 raft_consensus.cc:504] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:12.871483 29148 leader_election.cc:288] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [CANDIDATE]: Term 1 election: Requested vote from peers cf1f7a8afa5a42a7853b7ad1c2ee7855 (127.21.41.194:32937), 660fd2a7b01b4e15a201d53f7fe61e58 (127.21.41.193:38675)
I20260809 00:33:12.871690 28855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558ca78be93d45f1b60b66a17205693f" candidate_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855"
I20260809 00:33:12.871778 28855 raft_consensus.cc:3037] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:12.871728 28717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558ca78be93d45f1b60b66a17205693f" candidate_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "660fd2a7b01b4e15a201d53f7fe61e58"
I20260809 00:33:12.871846 28717 raft_consensus.cc:3037] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:12.872391 28855 raft_consensus.cc:2454] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ea8b2562624f53a4c81f0d3ee0be50 in term 1.
I20260809 00:33:12.872560 28717 raft_consensus.cc:2454] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ea8b2562624f53a4c81f0d3ee0be50 in term 1.
I20260809 00:33:12.872565 28926 leader_election.cc:302] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ea8b2562624f53a4c81f0d3ee0be50, cf1f7a8afa5a42a7853b7ad1c2ee7855; no voters:
I20260809 00:33:12.872731 29148 raft_consensus.cc:2781] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:12.872931 29148 raft_consensus.cc:686] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 1 LEADER]: Becoming Leader. State: Replica: 53ea8b2562624f53a4c81f0d3ee0be50, State: Running, Role: LEADER
I20260809 00:33:12.872993 29148 consensus_queue.cc:237] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } }
I20260809 00:33:12.873394 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.873227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.195:44713)
0809 00:33:12.873318 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.873359 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 00:33:12.873683 28584 catalog_manager.cc:5495] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ea8b2562624f53a4c81f0d3ee0be50 (127.21.41.195). New cstate: current_term: 1 leader_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ea8b2562624f53a4c81f0d3ee0be50" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 42291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 } health_report { overall_health: UNKNOWN } } }
I20260809 00:33:12.889933 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.887332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54656 (local address 127.21.41.195:42291)
0809 00:33:12.887401 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:12.887403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:12.887412 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:12.887438 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:12.887440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:12.887451 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:12.887500 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:33:12.887649 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.888185 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.889154 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:12.889385 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:12.889469 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:12.889657 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:33:12.889690 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:33:12.889748 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:12.889810 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 00:33:12.889852 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:33:12.890986 28717 raft_consensus.cc:1261] T 558ca78be93d45f1b60b66a17205693f P 660fd2a7b01b4e15a201d53f7fe61e58 [term 1 FOLLOWER]: Refusing update from remote peer 53ea8b2562624f53a4c81f0d3ee0be50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:12.891016 28855 raft_consensus.cc:1261] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 1 FOLLOWER]: Refusing update from remote peer 53ea8b2562624f53a4c81f0d3ee0be50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:12.891250 29150 consensus_queue.cc:1035] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "660fd2a7b01b4e15a201d53f7fe61e58" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:12.891417 29148 consensus_queue.cc:1035] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf1f7a8afa5a42a7853b7ad1c2ee7855" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 32937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:12.891922 29123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.891813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.193:33515)
0809 00:33:12.891859 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.891891 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 00:33:12.892243 29121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.892145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.194:40037)
0809 00:33:12.892187 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.892212 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 00:33:12.893333 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.893186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.195:39663)
0809 00:33:12.893279 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.893306 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 00:33:12.893899 29121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.893786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.194:51077)
0809 00:33:12.893850 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.893875 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 00:33:12.894150 28583 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:12.894240 28583 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:12.895089 29123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.895001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.193:38037)
0809 00:33:12.895036 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.895060 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 00:33:12.895429 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.895328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46427 (local address 127.21.41.195:55763)
0809 00:33:12.895366 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.895398 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:46427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 00:33:12.895797 28583 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a02a760b5dd940f4adcd6227da071be3: Sending DeleteTablet for 3 replicas of tablet 558ca78be93d45f1b60b66a17205693f
I20260809 00:33:12.896052 28972 tablet_service.cc:1514] Processing DeleteTablet for tablet 558ca78be93d45f1b60b66a17205693f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:12 UTC) from {username='oryx', principal='oryx/127.21.41.252@KRBTEST.COM'} at 127.0.0.1:54642
I20260809 00:33:12.896219 28835 tablet_service.cc:1514] Processing DeleteTablet for tablet 558ca78be93d45f1b60b66a17205693f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:12 UTC) from {username='oryx', principal='oryx/127.21.41.252@KRBTEST.COM'} at 127.0.0.1:44238
I20260809 00:33:12.896286 28697 tablet_service.cc:1514] Processing DeleteTablet for tablet 558ca78be93d45f1b60b66a17205693f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:12 UTC) from {username='oryx', principal='oryx/127.21.41.252@KRBTEST.COM'} at 127.0.0.1:36704
I20260809 00:33:12.896315 28696 tablet_service.cc:1514] Processing DeleteTablet for tablet 558ca78be93d45f1b60b66a17205693f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:12 UTC) from {username='oryx', principal='oryx/127.21.41.252@KRBTEST.COM'} at 127.0.0.1:36704
I20260809 00:33:12.896430 29163 tablet_replica.cc:330] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855: stopping tablet replica
I20260809 00:33:12.896468 29162 tablet_replica.cc:330] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50: stopping tablet replica
I20260809 00:33:12.896479 29163 raft_consensus.cc:2229] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:12.896472 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 28537
I20260809 00:33:12.896529 29163 raft_consensus.cc:2258] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:12.896533 29162 raft_consensus.cc:2229] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:33:12.897038 29162 raft_consensus.cc:2258] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:12.897360 29162 ts_tablet_manager.cc:1905] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:12.898972 29162 ts_tablet_manager.cc:1918] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:12.899011 29162 log.cc:1196] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/558ca78be93d45f1b60b66a17205693f
I20260809 00:33:12.899189 29162 ts_tablet_manager.cc:1939] T 558ca78be93d45f1b60b66a17205693f P 53ea8b2562624f53a4c81f0d3ee0be50: Deleting consensus metadata
I20260809 00:33:12.899479 28571 catalog_manager.cc:4841] TS 53ea8b2562624f53a4c81f0d3ee0be50 (127.21.41.195:42291): tablet 558ca78be93d45f1b60b66a17205693f (table test-table [id=2291298febd847b097caefda8a88f36b]) successfully deleted
I20260809 00:33:12.902230 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.902090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38675 (local address 127.0.0.1:36720)
0809 00:33:12.902134 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.902198 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:38675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 00:33:12.902240 29163 ts_tablet_manager.cc:1905] T 558ca78be93d45f1b60b66a17205693f P cf1f7a8afa5a42a7853b7ad1c2ee7855: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:33:12.902388 28570 catalog_manager.cc:4584] TS 660fd2a7b01b4e15a201d53f7fe61e58 (127.21.41.193:38675): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 558ca78be93d45f1b60b66a17205693f: Network error: Client connection negotiation failed: client connection to 127.21.41.193:38675: connect: Connection refused (error 111)
I20260809 00:33:12.902415 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 28771
I20260809 00:33:12.907512 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:12.907412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:32937 (local address 127.0.0.1:44254)
0809 00:33:12.907462 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:12.907491 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.194:32937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:33:12.907577 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 28908
I20260809 00:33:12.913398 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 28255
I20260809 00:33:12.918632 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 28443
2026-08-09T00:33:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6227 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:33:12 dist-test-slave-zpfg krb5kdc[29167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:33:12 dist-test-slave-zpfg krb5kdc[29167](info): set up 2 sockets
Aug 09 00:33:12 dist-test-slave-zpfg krb5kdc[29167](info): commencing operation
krb5kdc: starting...
W20260809 00:33:13.968500 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:33:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.254.keytab.
Entry for principal oryx/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.254.keytab.
I20260809 00:33:14.093367 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:46733
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.254.keytab
--principal=oryx/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:46733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:14.176085 29180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:14.176309 29180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:14.176331 29180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:14.177433 29180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:14.177465 29180 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 00:33:14.177479 29180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:14.177491 29180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:14.177505 29180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:14.177516 29180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:14.178998 29180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:46733
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.254.keytab
--principal=oryx/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:46733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:14.179206 29180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:14.179375 29180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:14.184245 29186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:14.184224 29188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:14.184394 29185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:14.184612 29180 server_base.cc:1034] running on GCE node
I20260809 00:33:14.184870 29180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:14.185161 29180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:14.186374 29180 hybrid_clock.cc:648] HybridClock initialized: now 1786235594186322 us; error 45 us; skew 500 ppm
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:14.192148 29180 init.cc:377] Logged in from keytab as oryx/127.21.41.254@KRBTEST.COM (short username oryx)
I20260809 00:33:14.192616 29180 webserver.cc:460] Webserver started at http://127.21.41.254:44151/ using document root <none> and password file <none>
I20260809 00:33:14.192860 29180 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:14.192906 29180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:14.193082 29180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:14.194176 29180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "165cc746ca43445d9a0aa0f9a4c1a6a0"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "e9251740d5a07ff8418468a28138616d"
server_key_iv: "c4be6ee7b2501c92235039e600e3b225"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.194573 29180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "165cc746ca43445d9a0aa0f9a4c1a6a0"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "e9251740d5a07ff8418468a28138616d"
server_key_iv: "c4be6ee7b2501c92235039e600e3b225"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.195950 29180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:14.196755 29195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:14.196943 29180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:14.197039 29180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "165cc746ca43445d9a0aa0f9a4c1a6a0"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "e9251740d5a07ff8418468a28138616d"
server_key_iv: "c4be6ee7b2501c92235039e600e3b225"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.197094 29180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:14.219836 29180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:14.222244 29180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:14.222433 29180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:14.227226 29180 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:46733
I20260809 00:33:14.227353 29257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:46733 every 8 connection(s)
I20260809 00:33:14.227669 29180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:14.228400 29258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:14.228587 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 29180
I20260809 00:33:14.228683 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:14.228904 21671 external_mini_cluster.cc:1428] Setting key c30f3d6aff8a55d26bae4288ab124b47
I20260809 00:33:14.231873 29258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0: Bootstrap starting.
I20260809 00:33:14.232640 29258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:14.232957 29258 log.cc:824] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:14.233748 29258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0: No bootstrap required, opened a new log
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:14.236411 29258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165cc746ca43445d9a0aa0f9a4c1a6a0" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46733 } }
I20260809 00:33:14.236590 29258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:14.236617 29258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 165cc746ca43445d9a0aa0f9a4c1a6a0, State: Initialized, Role: FOLLOWER
I20260809 00:33:14.236779 29258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165cc746ca43445d9a0aa0f9a4c1a6a0" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46733 } }
I20260809 00:33:14.236886 29258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:14.236922 29258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:14.236972 29258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:14.237596 29258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165cc746ca43445d9a0aa0f9a4c1a6a0" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46733 } }
I20260809 00:33:14.237720 29258 leader_election.cc:302] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 165cc746ca43445d9a0aa0f9a4c1a6a0; no voters:
I20260809 00:33:14.237937 29258 leader_election.cc:288] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:14.237972 29263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:14.238081 29263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [term 1 LEADER]: Becoming Leader. State: Replica: 165cc746ca43445d9a0aa0f9a4c1a6a0, State: Running, Role: LEADER
I20260809 00:33:14.238162 29263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165cc746ca43445d9a0aa0f9a4c1a6a0" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46733 } }
I20260809 00:33:14.238224 29258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:14.238502 29263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "165cc746ca43445d9a0aa0f9a4c1a6a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165cc746ca43445d9a0aa0f9a4c1a6a0" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46733 } } }
I20260809 00:33:14.238572 29263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:14.238690 29263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 165cc746ca43445d9a0aa0f9a4c1a6a0. Latest consensus state: current_term: 1 leader_uuid: "165cc746ca43445d9a0aa0f9a4c1a6a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165cc746ca43445d9a0aa0f9a4c1a6a0" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46733 } } }
I20260809 00:33:14.238747 29263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:14.238880 29268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:14.239449 29268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:14.240486 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.229466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.21.41.254:46733)
0809 00:33:14.229631 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.229641 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.229677 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:33:14.230117 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.230124 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.230172 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.230293 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:14.231097 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.232042 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.233121 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.233701 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.233824 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.236516 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:14.236520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:14.236520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:14.236530 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:14.238698 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.239138 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.239140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.239141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.239245 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.239481 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.239482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.239483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.239824 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:14.239988 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.240099 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.240238 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260809 00:33:14.242794 29268 catalog_manager.cc:1347] Generated new cluster ID: 4dd544bab96146e2a0b5cdeeca7b7c0e
I20260809 00:33:14.242835 29268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:33:14.255826 29268 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:14.256533 29268 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:14.264247 29268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0: Generated new TSK 0
I20260809 00:33:14.264474 29268 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab.
Entry for principal oryx/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab.
I20260809 00:33:14.301716 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46733
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab
--principal=oryx/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:14.393224 29284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:14.393430 29284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:14.393455 29284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:14.394702 29284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:14.394744 29284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:14.394765 29284 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:33:14.394819 29284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:33:14.396354 29284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab
--principal=oryx/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.41.254:46733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:14.396615 29284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:14.396857 29284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:14.402014 29289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:14.402006 29290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:14.402006 29292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:14.402274 29284 server_base.cc:1034] running on GCE node
I20260809 00:33:14.402530 29284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:14.402730 29284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:14.403957 29284 hybrid_clock.cc:648] HybridClock initialized: now 1786235594403934 us; error 26 us; skew 500 ppm
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:14.406796 29284 init.cc:377] Logged in from keytab as oryx/127.21.41.193@KRBTEST.COM (short username oryx)
I20260809 00:33:14.407284 29284 webserver.cc:460] Webserver started at http://127.21.41.193:36669/ using document root <none> and password file <none>
I20260809 00:33:14.407485 29284 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:14.407526 29284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:14.407621 29284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:14.408625 29284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "d78f2dd2181747a3b537106a7b59b663"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "6f5fe999cb9fa701379c81f80b5a8f0a"
server_key_iv: "cd42334510a24e0da08c70e2423f65db"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.408918 29284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "d78f2dd2181747a3b537106a7b59b663"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "6f5fe999cb9fa701379c81f80b5a8f0a"
server_key_iv: "cd42334510a24e0da08c70e2423f65db"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.410156 29284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:33:14.410992 29299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:14.411160 29284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:14.411266 29284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "d78f2dd2181747a3b537106a7b59b663"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "6f5fe999cb9fa701379c81f80b5a8f0a"
server_key_iv: "cd42334510a24e0da08c70e2423f65db"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.411342 29284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:14.447647 29284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:14.450098 29284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:14.450268 29284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:14.450829 29284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:14.450893 29284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:14.450941 29284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:14.450965 29284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:14.458140 29284 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:38609
I20260809 00:33:14.458209 29414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:38609 every 8 connection(s)
I20260809 00:33:14.458598 29284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:33:14.460954 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.451386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:36741 (local address 127.21.41.254:46733)
0809 00:33:14.451437 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.451439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.451691 (+ 252us) server_negotiation.cc:407] Connection header received
0809 00:33:14.452237 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.452239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.452251 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.452304 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:14.453173 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.454085 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.455218 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.455605 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.455692 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.458298 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:14.458299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:14.458299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:14.458308 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:14.459776 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.460162 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.460163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.460163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.460223 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.460436 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.460437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.460437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.460543 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:14.460594 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.460822 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.460890 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:33:14.461108 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.451395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46733 (local address 127.21.41.193:36741)
0809 00:33:14.451565 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.451602 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.452119 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.452379 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.452384 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.452758 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:14.453049 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.453056 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.454192 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.454193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.455111 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.455114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.455723 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.455725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.455863 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.456414 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:14.456428 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:14.458186 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:14.459874 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.459880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.459881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.460091 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.460285 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.460286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.460286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.460381 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.460596 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:14.460600 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:14.460708 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.460798 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.460917 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260809 00:33:14.461639 29415 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46733
I20260809 00:33:14.461725 29415 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:14.461915 29415 heartbeater.cc:511] Master 127.21.41.254:46733 requested a full tablet report, sending...
I20260809 00:33:14.462531 29211 ts_manager.cc:194] Registered new tserver with Master: d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609)
I20260809 00:33:14.463445 29211 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:36741
I20260809 00:33:14.467970 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 29284
I20260809 00:33:14.468104 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:33:14.468302 21671 external_mini_cluster.cc:1428] Setting key 4575c3b3e1b58d2b1db6abd22170a520
I20260809 00:33:14.468891 29421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.462784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46733 (local address 127.21.41.193:56023)
0809 00:33:14.462956 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.462966 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.463017 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.463150 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.463150 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.463299 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:14.463346 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.463348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.464140 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.464141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.464666 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.464667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.465086 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.465087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.465162 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.465531 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:14.465535 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:14.465918 (+ 383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:14.467707 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.467708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.467709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.467965 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.468198 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.468198 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.468199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.468262 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.468524 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:14.468525 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:14.468583 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.468677 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.468785 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":130,"threads_started":1}
I20260809 00:33:14.469259 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.462831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:56023 (local address 127.21.41.254:46733)
0809 00:33:14.462870 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.462871 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.463053 (+ 182us) server_negotiation.cc:407] Connection header received
0809 00:33:14.463083 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.463084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.463094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.463147 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:14.463411 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.464027 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.464763 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.465001 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.465094 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.466012 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:14.466013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:14.466013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:14.466021 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:14.467619 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.468053 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.468054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.468054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.468120 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.468340 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.468341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.468342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.468470 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:14.468532 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.469121 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.469204 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab.
Entry for principal oryx/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab.
I20260809 00:33:14.495198 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46733
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab
--principal=oryx/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:14.576486 29424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:14.576728 29424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:14.576768 29424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:14.577920 29424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:14.577988 29424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:14.578022 29424 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:33:14.578063 29424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:33:14.579595 29424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab
--principal=oryx/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.41.254:46733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:14.579890 29424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:14.580125 29424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:14.585103 29432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:14.585186 29429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:14.585194 29430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:14.585224 29424 server_base.cc:1034] running on GCE node
I20260809 00:33:14.585544 29424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:14.585790 29424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:14.586926 29424 hybrid_clock.cc:648] HybridClock initialized: now 1786235594586905 us; error 39 us; skew 500 ppm
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:14.589303 29424 init.cc:377] Logged in from keytab as oryx/127.21.41.194@KRBTEST.COM (short username oryx)
I20260809 00:33:14.589789 29424 webserver.cc:460] Webserver started at http://127.21.41.194:36597/ using document root <none> and password file <none>
I20260809 00:33:14.590027 29424 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:14.590095 29424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:14.590246 29424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:14.591323 29424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "8b0f73b49bf444bdb09e3cd15a4c611d"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "5432640cbda6275716c5aa4417ad285c"
server_key_iv: "aa24be263276104120eeb45d768cbd58"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.591686 29424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "8b0f73b49bf444bdb09e3cd15a4c611d"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "5432640cbda6275716c5aa4417ad285c"
server_key_iv: "aa24be263276104120eeb45d768cbd58"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.593003 29424 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:33:14.593777 29439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:14.593940 29424 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:14.594029 29424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "8b0f73b49bf444bdb09e3cd15a4c611d"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "5432640cbda6275716c5aa4417ad285c"
server_key_iv: "aa24be263276104120eeb45d768cbd58"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.594110 29424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:14.608415 29424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:14.610421 29424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:14.610586 29424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:14.611186 29424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:14.611245 29424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:14.611295 29424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:14.611323 29424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:14.618924 29424 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:41023
I20260809 00:33:14.619012 29554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:41023 every 8 connection(s)
I20260809 00:33:14.619361 29424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:33:14.620288 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 29424
I20260809 00:33:14.620389 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:33:14.620592 21671 external_mini_cluster.cc:1428] Setting key 7e184e26978c0d7d3cef806e3d870276
I20260809 00:33:14.622881 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.611655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:38009 (local address 127.21.41.254:46733)
0809 00:33:14.611731 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.611733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.611911 (+ 178us) server_negotiation.cc:407] Connection header received
0809 00:33:14.612420 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.612422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.612435 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.612514 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:14.613128 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.613988 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.615822 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.616037 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.616135 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.618673 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:14.618674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:14.618675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:14.618683 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:14.620297 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.620827 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.620828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.620829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.620893 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.621167 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.621168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.621168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.621340 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:14.621414 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.622690 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.622789 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:33:14.623078 29448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.611629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46733 (local address 127.21.41.194:38009)
0809 00:33:14.611778 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.611819 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.612321 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.612580 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.612584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.612829 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:14.613015 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.613019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.614146 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.614148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.615715 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.615720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.616160 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.616161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.616303 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.616843 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:14.616853 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:14.618566 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:14.620420 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.620427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.620429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.620711 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.620965 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.620966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.620967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.621091 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.622411 (+ 1320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:14.622417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:14.622547 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.622666 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.622830 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260809 00:33:14.623662 29555 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46733
I20260809 00:33:14.623740 29555 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:14.623927 29555 heartbeater.cc:511] Master 127.21.41.254:46733 requested a full tablet report, sending...
I20260809 00:33:14.624351 29211 ts_manager.cc:194] Registered new tserver with Master: 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023)
I20260809 00:33:14.624802 29211 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:38009
WARNING: no policy specified for oryx/127.21.41.195@KRBTEST.COM; defaulting to no policy
I20260809 00:33:14.630880 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.625132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:36421 (local address 127.21.41.254:46733)
0809 00:33:14.625188 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.625190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.625314 (+ 124us) server_negotiation.cc:407] Connection header received
0809 00:33:14.625337 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.625338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.625348 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.625415 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:14.625734 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.626282 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.627008 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.627198 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.627279 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.628283 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:14.628285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:14.628285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:14.628292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:14.629599 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.629948 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.629949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.629950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.630029 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.630242 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.630243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.630243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.630478 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:14.630558 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.630721 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.630790 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 00:33:14.630887 29562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.625098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46733 (local address 127.21.41.194:36421)
0809 00:33:14.625245 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.625255 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.625304 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.625417 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.625417 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.625571 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:14.625645 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.625646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.626394 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.626395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.626911 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.626913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.627281 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.627281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.627356 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.627874 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:14.627878 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:14.628144 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:14.629685 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.629686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.629686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.629861 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.630089 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.630090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.630090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.630161 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.630549 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:14.630550 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:14.630597 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.630686 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.630790 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab.
Entry for principal oryx/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab.
I20260809 00:33:14.649102 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46733
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab
--principal=oryx/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:14.734702 29564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:14.734900 29564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:14.734922 29564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:14.736078 29564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:14.736122 29564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:14.736186 29564 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:33:14.736222 29564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:14.738030 29564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab
--principal=oryx/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.41.254:46733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:14.738222 29564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:14.738466 29564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:14.743708 29570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:14.743772 29569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:14.743911 29564 server_base.cc:1034] running on GCE node
W20260809 00:33:14.743784 29572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:14.744151 29564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:14.744345 29564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:14.745509 29564 hybrid_clock.cc:648] HybridClock initialized: now 1786235594745498 us; error 30 us; skew 500 ppm
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:14.748025 29564 init.cc:377] Logged in from keytab as oryx/127.21.41.195@KRBTEST.COM (short username oryx)
I20260809 00:33:14.748521 29564 webserver.cc:460] Webserver started at http://127.21.41.195:44833/ using document root <none> and password file <none>
I20260809 00:33:14.748759 29564 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:14.748804 29564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:14.748952 29564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:14.749989 29564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "bc98ba70cd914f7ead7f8bc690e2dc94"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "09bf530567492f1e169d60143c1fe6cd"
server_key_iv: "1671042cbd979b9379e299bf4ac81fd4"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.750383 29564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "bc98ba70cd914f7ead7f8bc690e2dc94"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "09bf530567492f1e169d60143c1fe6cd"
server_key_iv: "1671042cbd979b9379e299bf4ac81fd4"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.751714 29564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:14.752429 29579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:14.752600 29564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:14.752697 29564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "bc98ba70cd914f7ead7f8bc690e2dc94"
format_stamp: "Formatted at 2026-08-09 00:33:14 on dist-test-slave-zpfg"
server_key: "09bf530567492f1e169d60143c1fe6cd"
server_key_iv: "1671042cbd979b9379e299bf4ac81fd4"
server_key_version: "encryptionkey@0"
I20260809 00:33:14.752787 29564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:14.770767 29564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:14.773063 29564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:14.773226 29564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:14.773841 29564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:14.773895 29564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:14.773938 29564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:14.773967 29564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:14.782253 29564 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:37459
I20260809 00:33:14.782759 29564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:14.783674 29694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:37459 every 8 connection(s)
I20260809 00:33:14.784509 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 29564
I20260809 00:33:14.784600 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:33:14.784814 21671 external_mini_cluster.cc:1428] Setting key 2395792f4d6305343cb74a3e1635cce7
I20260809 00:33:14.786770 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.774451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:48099 (local address 127.21.41.254:46733)
0809 00:33:14.774539 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.774542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.774854 (+ 312us) server_negotiation.cc:407] Connection header received
0809 00:33:14.775636 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.775637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.775650 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.775803 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:14.777135 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.777991 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.780219 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.780519 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.780593 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.783459 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:14.783460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:14.783461 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:14.783470 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:14.785142 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.785731 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.785732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.785733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.785804 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.786135 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.786136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.786136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.786276 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:14.786330 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.786640 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.786707 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 00:33:14.786993 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.774399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46733 (local address 127.21.41.195:48099)
0809 00:33:14.774681 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.774729 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.775494 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.775993 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.775998 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.776553 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:14.777015 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.777046 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.778116 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.778117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.780087 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.780091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.780663 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.780664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.780816 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.781422 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:14.781433 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:14.783315 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:14.785280 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.785286 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.785287 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.785497 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.785895 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.785896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.785896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.786054 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.786378 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:14.786385 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:14.786512 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.786595 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.786739 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":87,"threads_started":1}
I20260809 00:33:14.787467 29695 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46733
I20260809 00:33:14.787564 29695 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:14.787791 29695 heartbeater.cc:511] Master 127.21.41.254:46733 requested a full tablet report, sending...
I20260809 00:33:14.788230 29212 ts_manager.cc:194] Registered new tserver with Master: bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195:37459)
I20260809 00:33:14.788763 29212 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:48099
I20260809 00:33:14.793622 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.788556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:54607 (local address 127.21.41.254:46733)
0809 00:33:14.788597 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.788599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.788733 (+ 134us) server_negotiation.cc:407] Connection header received
0809 00:33:14.788799 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.788800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.788810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.788905 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:14.789264 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.789851 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.790582 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.790769 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.790829 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.791678 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:14.791678 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:14.791679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:14.791683 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:14.792753 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.793037 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.793037 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.793038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.793084 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.793237 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.793237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.793237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.793323 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:14.793417 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.793509 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.793565 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:33:14.793628 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.788469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46733 (local address 127.21.41.195:54607)
0809 00:33:14.788657 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.788667 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.788726 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.788910 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.788911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.789111 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:14.789187 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.789188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.789935 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.789936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.790495 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.790497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.790836 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.790837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.790884 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.791332 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:14.791343 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:14.791606 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:14.792817 (+ 1211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.792818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.792819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.792971 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.793138 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:14.793138 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:14.793138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:14.793186 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:14.793404 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:14.793405 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:14.793438 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.793488 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.793557 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":102,"threads_started":1}
I20260809 00:33:14.796988 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 00:33:14.818851 29261 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:14.813263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36542 (local address 127.21.41.254:46733)
0809 00:33:14.813340 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.813342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.813354 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:33:14.813383 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.813384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.813401 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.813466 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:14.814065 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.814681 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.815448 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.815659 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.815757 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.818752 (+ 2995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36542: BlockingRecv error: recv got EOF from 127.0.0.1:36542 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 00:33:14 dist-test-slave-zpfg krb5kdc[29167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:14.825876 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.819586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36556 (local address 127.21.41.254:46733)
0809 00:33:14.819656 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.819658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.819690 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:33:14.819749 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.819750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.819758 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.819804 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:14.820243 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.820687 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.821311 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.821480 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.821530 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.823753 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:14.823769 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:14.823769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:14.823775 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:14.824999 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.825295 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.825296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.825296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.825345 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:14.825502 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:14.825503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:14.825503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:14.825602 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:14.825674 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.825753 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.825810 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:33:14.827657 29212 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:33:14.829262 29212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:33:14.836156 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.831664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:37459 (local address 127.0.0.1:46960)
0809 00:33:14.832111 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.832157 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.832236 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.832548 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.832549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.832552 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:14.832628 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.832635 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.833965 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.833966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.835066 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.835068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.835942 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.835943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.836000 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.836010 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.836063 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.836099 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":29,"thread_start_us":100,"threads_started":1}
I20260809 00:33:14.836241 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.831809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.21.41.195:37459)
0809 00:33:14.832043 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.832048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.832241 (+ 193us) server_negotiation.cc:407] Connection header received
0809 00:33:14.832389 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.832392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.832405 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.832476 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:14.832735 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.833877 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.835158 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.835860 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.835938 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.836019 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.836086 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.836126 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":112,"threads_started":1}
I20260809 00:33:14.836550 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.831918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:41023 (local address 127.0.0.1:50630)
0809 00:33:14.832446 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.832454 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.832549 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.832851 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.832852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.832854 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:14.832907 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.832909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.833820 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.833822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.835330 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.835331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.836232 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.836233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.836398 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.836403 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.836445 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.836488 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":76,"thread_start_us":74,"threads_started":1}
I20260809 00:33:14.837025 29619 tablet_service.cc:1467] Processing CreateTablet for tablet 7bab1ae4d59e43cdbccfaeea1d31154d (DEFAULT_TABLE table=test-table [id=a8963f254d4940b6b4d4ac1735744dec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:14.837421 29619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bab1ae4d59e43cdbccfaeea1d31154d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:14.837635 29721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.831668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38609 (local address 127.0.0.1:60710)
0809 00:33:14.832127 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.832157 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.832235 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.832538 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.832541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.832543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:14.832619 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.832621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.833945 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.833946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.835324 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.835327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.837389 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.837390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.837467 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.837471 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.837530 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.837571 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":71,"thread_start_us":81,"threads_started":1}
I20260809 00:33:14.837643 29722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.832167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50630 (local address 127.21.41.194:41023)
0809 00:33:14.832669 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.832673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.832684 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:33:14.832731 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.832733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.832748 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.832843 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:14.832962 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.833690 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.835414 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.836146 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.836218 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.837462 (+ 1244us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.837510 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.837552 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":95,"threads_started":1}
I20260809 00:33:14.837972 29719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.831802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.21.41.193:38609)
0809 00:33:14.831994 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.832000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.832271 (+ 271us) server_negotiation.cc:407] Connection header received
0809 00:33:14.832338 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.832341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.832358 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.832444 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:14.832703 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.833857 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.836738 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.837301 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.837369 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.837417 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.837822 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.837865 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260809 00:33:14.838282 29489 tablet_service.cc:1467] Processing CreateTablet for tablet 7bab1ae4d59e43cdbccfaeea1d31154d (DEFAULT_TABLE table=test-table [id=a8963f254d4940b6b4d4ac1735744dec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:14.838681 29489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bab1ae4d59e43cdbccfaeea1d31154d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:14.840802 29723 tablet_bootstrap.cc:492] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94: Bootstrap starting.
I20260809 00:33:14.841475 29723 tablet_bootstrap.cc:654] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:14.841786 29723 log.cc:824] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:14.842111 29724 tablet_bootstrap.cc:492] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d: Bootstrap starting.
I20260809 00:33:14.842687 29723 tablet_bootstrap.cc:492] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94: No bootstrap required, opened a new log
I20260809 00:33:14.842680 29349 tablet_service.cc:1467] Processing CreateTablet for tablet 7bab1ae4d59e43cdbccfaeea1d31154d (DEFAULT_TABLE table=test-table [id=a8963f254d4940b6b4d4ac1735744dec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:14.842774 29723 ts_tablet_manager.cc:1397] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:33:14.842932 29349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bab1ae4d59e43cdbccfaeea1d31154d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:14.843052 29724 tablet_bootstrap.cc:654] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:14.843436 29724 log.cc:824] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:14.844321 29724 tablet_bootstrap.cc:492] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d: No bootstrap required, opened a new log
I20260809 00:33:14.844435 29724 ts_tablet_manager.cc:1397] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:33:14.844853 29723 raft_consensus.cc:348] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } }
I20260809 00:33:14.845031 29723 raft_consensus.cc:374] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:14.845077 29723 raft_consensus.cc:729] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc98ba70cd914f7ead7f8bc690e2dc94, State: Initialized, Role: FOLLOWER
I20260809 00:33:14.845196 29723 consensus_queue.cc:260] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } }
I20260809 00:33:14.845513 29695 heartbeater.cc:503] Master 127.21.41.254:46733 was elected leader, sending a full tablet report...
I20260809 00:33:14.845767 29723 ts_tablet_manager.cc:1428] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:33:14.846221 29728 tablet_bootstrap.cc:492] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663: Bootstrap starting.
I20260809 00:33:14.846841 29728 tablet_bootstrap.cc:654] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:14.847143 29728 log.cc:824] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:14.847180 29724 raft_consensus.cc:348] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } }
I20260809 00:33:14.847368 29724 raft_consensus.cc:374] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:14.847419 29724 raft_consensus.cc:729] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b0f73b49bf444bdb09e3cd15a4c611d, State: Initialized, Role: FOLLOWER
I20260809 00:33:14.847554 29724 consensus_queue.cc:260] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } }
I20260809 00:33:14.847864 29724 ts_tablet_manager.cc:1428] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:33:14.847913 29555 heartbeater.cc:503] Master 127.21.41.254:46733 was elected leader, sending a full tablet report...
I20260809 00:33:14.847956 29728 tablet_bootstrap.cc:492] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663: No bootstrap required, opened a new log
I20260809 00:33:14.848021 29728 ts_tablet_manager.cc:1397] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:33:14.848999 29727 raft_consensus.cc:482] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:33:14.849078 29727 raft_consensus.cc:504] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } }
I20260809 00:33:14.849399 29727 leader_election.cc:288] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609), 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023)
I20260809 00:33:14.849836 29728 raft_consensus.cc:348] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } }
I20260809 00:33:14.849987 29728 raft_consensus.cc:374] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:14.850024 29728 raft_consensus.cc:729] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78f2dd2181747a3b537106a7b59b663, State: Initialized, Role: FOLLOWER
I20260809 00:33:14.850145 29728 consensus_queue.cc:260] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } }
I20260809 00:33:14.850440 29728 ts_tablet_manager.cc:1428] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:33:14.850481 29415 heartbeater.cc:503] Master 127.21.41.254:46733 was elected leader, sending a full tablet report...
I20260809 00:33:14.852620 29719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.849597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:34221 (local address 127.21.41.193:38609)
0809 00:33:14.849742 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.849744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.849752 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:33:14.849777 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.849778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.849787 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.849839 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:14.849984 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.850750 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.851867 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.852233 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.852329 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.852338 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.852519 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.852553 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 00:33:14.852635 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.849532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38609 (local address 127.21.41.195:34221)
0809 00:33:14.849581 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.849592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.849631 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.849840 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.849840 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.849842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:14.849909 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.849911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.850840 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.850841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.851765 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.851767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.852329 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.852330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.852431 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.852434 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.852511 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.852553 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 00:33:14.852764 29722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.849904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:57821 (local address 127.21.41.194:41023)
0809 00:33:14.849972 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.849975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.849984 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:14.850007 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.850008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.850016 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.850077 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:14.850320 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.851063 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.852046 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.852423 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.852507 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.852515 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.852661 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.852697 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:33:14.852784 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.849543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:41023 (local address 127.21.41.195:57821)
0809 00:33:14.849714 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.849723 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.849761 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.850127 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.850128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.850130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:14.850197 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.850199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.851134 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.851135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.851955 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.851957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.852507 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.852508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.852579 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.852582 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.852657 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.852695 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":82,"threads_started":1}
I20260809 00:33:14.853055 29509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bab1ae4d59e43cdbccfaeea1d31154d" candidate_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" is_pre_election: true
I20260809 00:33:14.853080 29369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bab1ae4d59e43cdbccfaeea1d31154d" candidate_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78f2dd2181747a3b537106a7b59b663" is_pre_election: true
I20260809 00:33:14.853154 29509 raft_consensus.cc:2454] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc98ba70cd914f7ead7f8bc690e2dc94 in term 0.
I20260809 00:33:14.853184 29369 raft_consensus.cc:2454] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc98ba70cd914f7ead7f8bc690e2dc94 in term 0.
I20260809 00:33:14.853348 29580 leader_election.cc:302] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b0f73b49bf444bdb09e3cd15a4c611d, bc98ba70cd914f7ead7f8bc690e2dc94; no voters:
I20260809 00:33:14.853454 29727 raft_consensus.cc:2781] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:33:14.853519 29727 raft_consensus.cc:482] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:33:14.853559 29727 raft_consensus.cc:3037] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:14.854513 29727 raft_consensus.cc:504] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } }
I20260809 00:33:14.854664 29727 leader_election.cc:288] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [CANDIDATE]: Term 1 election: Requested vote from peers d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609), 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023)
I20260809 00:33:14.854794 29369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bab1ae4d59e43cdbccfaeea1d31154d" candidate_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78f2dd2181747a3b537106a7b59b663"
I20260809 00:33:14.854815 29509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bab1ae4d59e43cdbccfaeea1d31154d" candidate_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d"
I20260809 00:33:14.854867 29369 raft_consensus.cc:3037] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:14.854919 29509 raft_consensus.cc:3037] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:14.855546 29369 raft_consensus.cc:2454] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc98ba70cd914f7ead7f8bc690e2dc94 in term 1.
I20260809 00:33:14.855758 29582 leader_election.cc:302] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc98ba70cd914f7ead7f8bc690e2dc94, d78f2dd2181747a3b537106a7b59b663; no voters:
I20260809 00:33:14.855863 29509 raft_consensus.cc:2454] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc98ba70cd914f7ead7f8bc690e2dc94 in term 1.
I20260809 00:33:14.855919 29727 raft_consensus.cc:2781] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:14.856076 29727 raft_consensus.cc:686] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 LEADER]: Becoming Leader. State: Replica: bc98ba70cd914f7ead7f8bc690e2dc94, State: Running, Role: LEADER
I20260809 00:33:14.856220 29727 consensus_queue.cc:237] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } }
I20260809 00:33:14.857041 29212 catalog_manager.cc:5495] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195). New cstate: current_term: 1 leader_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } health_report { overall_health: UNKNOWN } } }
I20260809 00:33:14.864890 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.862759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46966 (local address 127.21.41.195:37459)
0809 00:33:14.862820 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.862822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.862831 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:14.862870 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.862871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.862887 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.862945 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:33:14.863054 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.863505 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.864203 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.864396 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.864450 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.864587 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:33:14.864681 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:33:14.864747 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.864769 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.864787 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:33:14.866654 29369 raft_consensus.cc:1261] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Refusing update from remote peer bc98ba70cd914f7ead7f8bc690e2dc94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:14.866654 29509 raft_consensus.cc:1261] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Refusing update from remote peer bc98ba70cd914f7ead7f8bc690e2dc94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:14.867014 29727 consensus_queue.cc:1035] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:14.867170 29733 consensus_queue.cc:1035] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:14.869902 29737 mvcc.cc:205] Tried to move back new op lower bound from 7316420996571222016 to 7316420996531904512. Current Snapshot: MvccSnapshot[applied={T|T < 7316420996571222016}]
I20260809 00:33:14.872929 29212 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:14.873049 29212 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:14.873945 29212 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0: Sending DeleteTablet for 3 replicas of tablet 7bab1ae4d59e43cdbccfaeea1d31154d
I20260809 00:33:14.874294 29349 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bab1ae4d59e43cdbccfaeea1d31154d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:14 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:60710
I20260809 00:33:14.874446 29619 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bab1ae4d59e43cdbccfaeea1d31154d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:14 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:46960
I20260809 00:33:14.874444 29489 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bab1ae4d59e43cdbccfaeea1d31154d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:14 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:50630
I20260809 00:33:14.874583 29212 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:33:14.874734 29212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:33:14.874784 29747 tablet_replica.cc:330] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94: stopping tablet replica
I20260809 00:33:14.874878 29747 raft_consensus.cc:2229] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:33:14.875005 29747 raft_consensus.cc:2258] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:14.875175 29746 tablet_replica.cc:330] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663: stopping tablet replica
I20260809 00:33:14.875294 29746 raft_consensus.cc:2229] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:14.875412 29746 raft_consensus.cc:2258] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:14.875835 29748 tablet_replica.cc:330] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d: stopping tablet replica
I20260809 00:33:14.876086 29489 tablet_service.cc:1467] Processing CreateTablet for tablet 566bbdd2f2734fc0afe7104ba9ff7374 (DEFAULT_TABLE table=test-table [id=f34bb968dc6245f0abb29667885d06a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:14.876101 29349 tablet_service.cc:1467] Processing CreateTablet for tablet 566bbdd2f2734fc0afe7104ba9ff7374 (DEFAULT_TABLE table=test-table [id=f34bb968dc6245f0abb29667885d06a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:14.876111 29619 tablet_service.cc:1467] Processing CreateTablet for tablet 566bbdd2f2734fc0afe7104ba9ff7374 (DEFAULT_TABLE table=test-table [id=f34bb968dc6245f0abb29667885d06a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:14.876220 29349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 566bbdd2f2734fc0afe7104ba9ff7374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:14.876228 29489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 566bbdd2f2734fc0afe7104ba9ff7374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:14.876230 29619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 566bbdd2f2734fc0afe7104ba9ff7374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:14.877696 29724 tablet_bootstrap.cc:492] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d: Bootstrap starting.
I20260809 00:33:14.877928 29728 tablet_bootstrap.cc:492] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663: Bootstrap starting.
I20260809 00:33:14.877956 29723 tablet_bootstrap.cc:492] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94: Bootstrap starting.
I20260809 00:33:14.878152 29724 tablet_bootstrap.cc:654] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:14.878403 29728 tablet_bootstrap.cc:654] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:14.878409 29723 tablet_bootstrap.cc:654] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:14.879148 29746 ts_tablet_manager.cc:1905] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:14.879324 29747 ts_tablet_manager.cc:1905] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:14.879369 29723 tablet_bootstrap.cc:492] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94: No bootstrap required, opened a new log
I20260809 00:33:14.879400 29723 ts_tablet_manager.cc:1397] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:14.879545 29723 raft_consensus.cc:348] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:14.879626 29723 raft_consensus.cc:374] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:14.879647 29723 raft_consensus.cc:729] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc98ba70cd914f7ead7f8bc690e2dc94, State: Initialized, Role: FOLLOWER
I20260809 00:33:14.879702 29723 consensus_queue.cc:260] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:14.879797 29723 ts_tablet_manager.cc:1428] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:14.880637 29748 raft_consensus.cc:2229] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:14.880791 29748 raft_consensus.cc:2258] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:14.880956 29747 ts_tablet_manager.cc:1918] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:14.881059 29747 log.cc:1196] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/7bab1ae4d59e43cdbccfaeea1d31154d
I20260809 00:33:14.881258 29724 tablet_bootstrap.cc:492] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d: No bootstrap required, opened a new log
I20260809 00:33:14.881316 29724 ts_tablet_manager.cc:1397] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260809 00:33:14.881346 29747 ts_tablet_manager.cc:1939] T 7bab1ae4d59e43cdbccfaeea1d31154d P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting consensus metadata
I20260809 00:33:14.881461 29724 raft_consensus.cc:348] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:14.881520 29724 raft_consensus.cc:374] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:14.881508 29728 tablet_bootstrap.cc:492] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663: No bootstrap required, opened a new log
I20260809 00:33:14.881532 29724 raft_consensus.cc:729] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b0f73b49bf444bdb09e3cd15a4c611d, State: Initialized, Role: FOLLOWER
I20260809 00:33:14.881548 29728 ts_tablet_manager.cc:1397] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260809 00:33:14.881594 29724 consensus_queue.cc:260] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:14.881682 29724 ts_tablet_manager.cc:1428] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:14.881752 29746 ts_tablet_manager.cc:1918] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:14.881711 29728 raft_consensus.cc:348] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:14.881807 29728 raft_consensus.cc:374] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:14.881839 29728 raft_consensus.cc:729] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78f2dd2181747a3b537106a7b59b663, State: Initialized, Role: FOLLOWER
I20260809 00:33:14.881878 29728 consensus_queue.cc:260] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:14.881860 29196 catalog_manager.cc:4841] TS bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195:37459): tablet 7bab1ae4d59e43cdbccfaeea1d31154d (table test-table [id=a8963f254d4940b6b4d4ac1735744dec]) successfully deleted
I20260809 00:33:14.881963 29728 ts_tablet_manager.cc:1428] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:14.881807 29746 log.cc:1196] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/wal/wals/7bab1ae4d59e43cdbccfaeea1d31154d
I20260809 00:33:14.882293 29746 ts_tablet_manager.cc:1939] T 7bab1ae4d59e43cdbccfaeea1d31154d P d78f2dd2181747a3b537106a7b59b663: Deleting consensus metadata
I20260809 00:33:14.882453 29748 ts_tablet_manager.cc:1905] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:14.882731 29198 catalog_manager.cc:4841] TS d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609): tablet 7bab1ae4d59e43cdbccfaeea1d31154d (table test-table [id=a8963f254d4940b6b4d4ac1735744dec]) successfully deleted
I20260809 00:33:14.883977 29748 ts_tablet_manager.cc:1918] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:14.884064 29748 log.cc:1196] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/7bab1ae4d59e43cdbccfaeea1d31154d
I20260809 00:33:14.884289 29748 ts_tablet_manager.cc:1939] T 7bab1ae4d59e43cdbccfaeea1d31154d P 8b0f73b49bf444bdb09e3cd15a4c611d: Deleting consensus metadata
I20260809 00:33:14.884655 29196 catalog_manager.cc:4841] TS 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023): tablet 7bab1ae4d59e43cdbccfaeea1d31154d (table test-table [id=a8963f254d4940b6b4d4ac1735744dec]) successfully deleted
W20260809 00:33:14.959350 29416 tablet.cc:2369] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:33:14.970105 29732 raft_consensus.cc:482] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:33:14.970225 29732 raft_consensus.cc:504] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:14.970741 29732 leader_election.cc:288] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023), bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195:37459)
I20260809 00:33:14.973940 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.970945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:41023 (local address 127.21.41.193:37457)
0809 00:33:14.971067 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.971080 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.971136 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.971518 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.971519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.971522 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:14.971602 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.971605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.972275 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.972275 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.973015 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.973016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.973611 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.973612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.973717 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.973722 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.973807 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.973854 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
I20260809 00:33:14.973954 29722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.971283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:37457 (local address 127.21.41.194:41023)
0809 00:33:14.971360 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.971362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.971371 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:14.971399 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.971400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.971408 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.971460 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:14.971688 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.972190 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.973106 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.973504 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.973575 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.973585 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.973840 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.973887 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:33:14.974493 29509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566bbdd2f2734fc0afe7104ba9ff7374" candidate_uuid: "d78f2dd2181747a3b537106a7b59b663" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" is_pre_election: true
I20260809 00:33:14.974606 29509 raft_consensus.cc:2454] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d78f2dd2181747a3b537106a7b59b663 in term 0.
I20260809 00:33:14.974632 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.971257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:50935 (local address 127.21.41.195:37459)
0809 00:33:14.971304 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:14.971306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:14.971319 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:33:14.971444 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:14.971445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:14.971456 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:14.971530 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:14.971753 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.972395 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.973694 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.974269 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.974371 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.974379 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:14.974520 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 00:33:14.974560 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:33:14.974752 29753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:14.971158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:37459 (local address 127.21.41.193:50935)
0809 00:33:14.971272 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:14.971282 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:14.971327 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:14.971582 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:14.971583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:14.971586 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:14.971668 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.971671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.972500 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.972501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.973590 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:14.973603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:14.974369 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:14.974370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:14.974449 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:14.974454 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:33:14.974510 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:33:14.974548 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":37,"threads_started":1}
I20260809 00:33:14.974831 29300 leader_election.cc:302] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b0f73b49bf444bdb09e3cd15a4c611d, d78f2dd2181747a3b537106a7b59b663; no voters:
I20260809 00:33:14.974969 29732 raft_consensus.cc:2781] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:33:14.975011 29732 raft_consensus.cc:482] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:33:14.975036 29732 raft_consensus.cc:3037] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:14.975198 29638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566bbdd2f2734fc0afe7104ba9ff7374" candidate_uuid: "d78f2dd2181747a3b537106a7b59b663" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" is_pre_election: true
I20260809 00:33:14.975312 29638 raft_consensus.cc:2454] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d78f2dd2181747a3b537106a7b59b663 in term 0.
I20260809 00:33:14.975909 29732 raft_consensus.cc:504] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:14.976039 29732 leader_election.cc:288] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [CANDIDATE]: Term 1 election: Requested vote from peers 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023), bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195:37459)
I20260809 00:33:14.976213 29509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566bbdd2f2734fc0afe7104ba9ff7374" candidate_uuid: "d78f2dd2181747a3b537106a7b59b663" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d"
I20260809 00:33:14.976224 29638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566bbdd2f2734fc0afe7104ba9ff7374" candidate_uuid: "d78f2dd2181747a3b537106a7b59b663" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94"
I20260809 00:33:14.976295 29509 raft_consensus.cc:3037] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:14.976315 29638 raft_consensus.cc:3037] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:14.976954 29638 raft_consensus.cc:2454] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d78f2dd2181747a3b537106a7b59b663 in term 1.
I20260809 00:33:14.977077 29509 raft_consensus.cc:2454] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d78f2dd2181747a3b537106a7b59b663 in term 1.
I20260809 00:33:14.977116 29300 leader_election.cc:302] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc98ba70cd914f7ead7f8bc690e2dc94, d78f2dd2181747a3b537106a7b59b663; no voters:
I20260809 00:33:14.977200 29732 raft_consensus.cc:2781] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:14.977327 29732 raft_consensus.cc:686] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 1 LEADER]: Becoming Leader. State: Replica: d78f2dd2181747a3b537106a7b59b663, State: Running, Role: LEADER
I20260809 00:33:14.977399 29732 consensus_queue.cc:237] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:14.978272 29212 catalog_manager.cc:5495] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 reported cstate change: term changed from 0 to 1, leader changed from <none> to d78f2dd2181747a3b537106a7b59b663 (127.21.41.193). New cstate: current_term: 1 leader_uuid: "d78f2dd2181747a3b537106a7b59b663" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } health_report { overall_health: UNKNOWN } } }
Aug 09 00:33:15 dist-test-slave-zpfg krb5kdc[29167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235594, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.254@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:15.018770 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.011167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36560 (local address 127.21.41.254:46733)
0809 00:33:15.011237 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:15.011239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:15.011398 (+ 159us) server_negotiation.cc:407] Connection header received
0809 00:33:15.011504 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:15.011505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:15.011514 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:15.011582 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:15.012120 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.012612 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.013303 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.013495 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.013580 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.016051 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:15.016053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:15.016053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:15.016060 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:15.017534 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:15.017931 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:15.017932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:15.017933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:15.017988 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:15.018183 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:15.018184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:15.018184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:15.018297 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:15.018398 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:15.018551 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 00:33:15.018660 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:33:15.018891 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.011132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46733 (local address 127.0.0.1:36560)
0809 00:33:15.011323 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:15.011333 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:15.011419 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:15.011587 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:15.011588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:15.011957 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:15.012009 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.012011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.012694 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.012695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.013222 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.013223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.013581 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.013582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.013646 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.014213 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:15.014218 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:15.015947 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:15.017634 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:15.017642 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:15.017644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:15.017842 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:15.018056 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:15.018056 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:15.018057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:15.018119 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:15.018382 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:15.018391 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:15.018437 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 00:33:15.018530 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 00:33:15.018745 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":111,"thread_start_us":113,"threads_started":1}
I20260809 00:33:15.020814 29212 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:36560:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 00:33:15.021025 29212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:33:15.022503 29349 tablet_service.cc:1467] Processing CreateTablet for tablet 9b5c3473e54c43bc8364380672caa4c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c6a457dfc0045d0bba41deb12944dfe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:33:15.022524 29489 tablet_service.cc:1467] Processing CreateTablet for tablet 9b5c3473e54c43bc8364380672caa4c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c6a457dfc0045d0bba41deb12944dfe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:33:15.022656 29619 tablet_service.cc:1467] Processing CreateTablet for tablet 9b5c3473e54c43bc8364380672caa4c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c6a457dfc0045d0bba41deb12944dfe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:33:15.022723 29349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b5c3473e54c43bc8364380672caa4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:15.022725 29489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b5c3473e54c43bc8364380672caa4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:15.022804 29619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b5c3473e54c43bc8364380672caa4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:15.024253 29723 tablet_bootstrap.cc:492] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94: Bootstrap starting.
I20260809 00:33:15.024255 29724 tablet_bootstrap.cc:492] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: Bootstrap starting.
I20260809 00:33:15.024576 29723 tablet_bootstrap.cc:654] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:15.024708 29724 tablet_bootstrap.cc:654] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:15.025080 29728 tablet_bootstrap.cc:492] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663: Bootstrap starting.
I20260809 00:33:15.025171 29723 tablet_bootstrap.cc:492] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94: No bootstrap required, opened a new log
I20260809 00:33:15.025211 29723 ts_tablet_manager.cc:1397] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:15.025331 29723 raft_consensus.cc:348] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.025396 29723 raft_consensus.cc:374] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:15.025421 29723 raft_consensus.cc:729] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc98ba70cd914f7ead7f8bc690e2dc94, State: Initialized, Role: FOLLOWER
I20260809 00:33:15.025427 29724 tablet_bootstrap.cc:492] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: No bootstrap required, opened a new log
I20260809 00:33:15.025461 29724 ts_tablet_manager.cc:1397] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:15.025480 29723 consensus_queue.cc:260] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.025509 29728 tablet_bootstrap.cc:654] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:15.025594 29723 ts_tablet_manager.cc:1428] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:15.025640 29724 raft_consensus.cc:348] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.025624 29743 tablet_replica.cc:439] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } }
I20260809 00:33:15.025699 29724 raft_consensus.cc:374] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:15.025723 29724 raft_consensus.cc:729] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b0f73b49bf444bdb09e3cd15a4c611d, State: Initialized, Role: FOLLOWER
I20260809 00:33:15.025712 29743 tablet_replica.cc:442] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:33:15.025772 29724 consensus_queue.cc:260] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.025856 29724 ts_tablet_manager.cc:1428] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:15.025962 29730 tablet_replica.cc:439] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } }
I20260809 00:33:15.026072 29730 tablet_replica.cc:442] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:33:15.026523 29728 tablet_bootstrap.cc:492] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663: No bootstrap required, opened a new log
I20260809 00:33:15.026595 29728 ts_tablet_manager.cc:1397] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:33:15.026742 29728 raft_consensus.cc:348] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.026818 29728 raft_consensus.cc:374] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:15.026844 29728 raft_consensus.cc:729] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78f2dd2181747a3b537106a7b59b663, State: Initialized, Role: FOLLOWER
I20260809 00:33:15.026901 29728 consensus_queue.cc:260] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.027006 29728 ts_tablet_manager.cc:1428] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:15.027077 29732 tablet_replica.cc:439] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } }
I20260809 00:33:15.027170 29732 tablet_replica.cc:442] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 00:33:15.033277 29696 tablet.cc:2369] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:33:15.120290 29556 tablet.cc:2369] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:33:15.374529 29732 consensus_queue.cc:1035] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:15.376837 29767 consensus_queue.cc:1035] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:15.399220 29730 raft_consensus.cc:482] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:33:15.399396 29730 raft_consensus.cc:504] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.399794 29730 leader_election.cc:288] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609), bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195:37459)
I20260809 00:33:15.403928 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.400124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:34551 (local address 127.21.41.193:38609)
0809 00:33:15.400241 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:15.400243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:15.400251 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:33:15.400296 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:15.400297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:15.400307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:15.400402 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:15.400581 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.401344 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.402971 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.403552 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.403627 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.403638 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:15.403797 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 00:33:15.403835 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
I20260809 00:33:15.403915 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.399995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38609 (local address 127.21.41.194:34551)
0809 00:33:15.400121 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:15.400132 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:15.400191 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:15.400421 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:15.400422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:15.400423 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:15.400476 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.400477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.401435 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.401436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.402872 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.402875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.403631 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.403632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.403715 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.403720 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:33:15.403775 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 00:33:15.403822 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":99,"thread_start_us":59,"threads_started":1}
I20260809 00:33:15.403999 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.400123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:50415 (local address 127.21.41.195:37459)
0809 00:33:15.400174 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:15.400176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:15.400189 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:33:15.400262 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:15.400263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:15.400275 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:15.400369 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:15.400580 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.401343 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.402969 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.403537 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.403714 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.403720 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:15.403881 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 00:33:15.403925 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:33:15.404012 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.399993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:37459 (local address 127.21.41.194:50415)
0809 00:33:15.400119 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:15.400132 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:15.400191 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:15.400372 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:15.400373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:15.400392 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:15.400456 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.400457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.401433 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.401434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.402873 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.402875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.403720 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.403720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.403808 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.403810 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:33:15.403863 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:33:15.403894 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":67,"thread_start_us":56,"threads_started":1}
I20260809 00:33:15.404202 29638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b5c3473e54c43bc8364380672caa4c6" candidate_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" is_pre_election: true
I20260809 00:33:15.404326 29638 raft_consensus.cc:2454] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b0f73b49bf444bdb09e3cd15a4c611d in term 0.
I20260809 00:33:15.404419 29369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b5c3473e54c43bc8364380672caa4c6" candidate_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78f2dd2181747a3b537106a7b59b663" is_pre_election: true
I20260809 00:33:15.404517 29369 raft_consensus.cc:2454] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b0f73b49bf444bdb09e3cd15a4c611d in term 0.
I20260809 00:33:15.404572 29440 leader_election.cc:302] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b0f73b49bf444bdb09e3cd15a4c611d, bc98ba70cd914f7ead7f8bc690e2dc94; no voters:
I20260809 00:33:15.404721 29730 raft_consensus.cc:2781] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:33:15.404775 29730 raft_consensus.cc:482] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:33:15.404798 29730 raft_consensus.cc:3037] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:15.405490 29730 raft_consensus.cc:504] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.405644 29730 leader_election.cc:288] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [CANDIDATE]: Term 1 election: Requested vote from peers d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609), bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195:37459)
I20260809 00:33:15.405810 29638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b5c3473e54c43bc8364380672caa4c6" candidate_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94"
I20260809 00:33:15.405869 29369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b5c3473e54c43bc8364380672caa4c6" candidate_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78f2dd2181747a3b537106a7b59b663"
I20260809 00:33:15.405911 29638 raft_consensus.cc:3037] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:15.405928 29369 raft_consensus.cc:3037] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:15.406695 29369 raft_consensus.cc:2454] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b0f73b49bf444bdb09e3cd15a4c611d in term 1.
I20260809 00:33:15.406744 29638 raft_consensus.cc:2454] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b0f73b49bf444bdb09e3cd15a4c611d in term 1.
I20260809 00:33:15.406880 29442 leader_election.cc:302] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b0f73b49bf444bdb09e3cd15a4c611d, d78f2dd2181747a3b537106a7b59b663; no voters:
I20260809 00:33:15.406986 29730 raft_consensus.cc:2781] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:15.407110 29730 raft_consensus.cc:686] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 LEADER]: Becoming Leader. State: Replica: 8b0f73b49bf444bdb09e3cd15a4c611d, State: Running, Role: LEADER
I20260809 00:33:15.407207 29730 consensus_queue.cc:237] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.407523 29774 tablet_replica.cc:439] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: TxnStatusTablet state changed. Reason: New leader 8b0f73b49bf444bdb09e3cd15a4c611d. Latest consensus state: current_term: 1 leader_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } }
I20260809 00:33:15.407629 29774 tablet_replica.cc:442] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:33:15.407809 29776 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 00:33:15.407948 29776 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 00:33:15.407980 29212 catalog_manager.cc:5495] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194). New cstate: current_term: 1 leader_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } health_report { overall_health: UNKNOWN } } }
I20260809 00:33:15.469132 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.466544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:41023 (local address 127.0.0.1:50644)
0809 00:33:15.466625 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:15.466640 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:15.466696 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:15.466892 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:15.466893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:15.466894 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:33:15.466955 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.466957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.467593 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.467595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.468203 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.468205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.468586 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.468587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.468660 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.468675 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:33:15.468950 (+ 275us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:33:15.468960 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 00:33:15.469007 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 00:33:15.469062 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 00:33:15.469123 29722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.466615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50644 (local address 127.21.41.194:41023)
0809 00:33:15.466670 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:15.466673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:15.466684 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:33:15.466784 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:15.466785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:15.466804 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:15.466885 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:33:15.467030 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.467510 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.468305 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.468508 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.468586 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.468777 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:33:15.468878 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:33:15.468964 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:15.469020 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 00:33:15.469039 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 00:33:15.470281 29369 raft_consensus.cc:1261] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Refusing update from remote peer 8b0f73b49bf444bdb09e3cd15a4c611d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:15.470361 29638 raft_consensus.cc:1261] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Refusing update from remote peer 8b0f73b49bf444bdb09e3cd15a4c611d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:15.470618 29774 consensus_queue.cc:1035] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:15.470796 29730 consensus_queue.cc:1035] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:15.471385 29767 tablet_replica.cc:439] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663: TxnStatusTablet state changed. Reason: New leader 8b0f73b49bf444bdb09e3cd15a4c611d. Latest consensus state: current_term: 1 leader_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } }
I20260809 00:33:15.471488 29767 tablet_replica.cc:442] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:33:15.471732 29743 tablet_replica.cc:439] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94: TxnStatusTablet state changed. Reason: New leader 8b0f73b49bf444bdb09e3cd15a4c611d. Latest consensus state: current_term: 1 leader_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } }
I20260809 00:33:15.471827 29743 tablet_replica.cc:442] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:33:15.472361 29730 tablet_replica.cc:439] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } }
I20260809 00:33:15.472453 29730 tablet_replica.cc:442] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:33:15.472605 29730 tablet_replica.cc:439] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } }
I20260809 00:33:15.472673 29730 tablet_replica.cc:442] T 9b5c3473e54c43bc8364380672caa4c6 P 8b0f73b49bf444bdb09e3cd15a4c611d: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:33:15.472999 29767 tablet_replica.cc:439] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } }
I20260809 00:33:15.473086 29767 tablet_replica.cc:442] T 9b5c3473e54c43bc8364380672caa4c6 P d78f2dd2181747a3b537106a7b59b663: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:33:15.473065 29766 mvcc.cc:205] Tried to move back new op lower bound from 7316420999044091904 to 7316420998788435968. Current Snapshot: MvccSnapshot[applied={T|T < 7316420999044091904}]
I20260809 00:33:15.473344 29768 mvcc.cc:205] Tried to move back new op lower bound from 7316420999044091904 to 7316420998788435968. Current Snapshot: MvccSnapshot[applied={T|T < 7316420999044091904}]
I20260809 00:33:15.473425 29743 tablet_replica.cc:439] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } } }
I20260809 00:33:15.473510 29743 tablet_replica.cc:442] T 9b5c3473e54c43bc8364380672caa4c6 P bc98ba70cd914f7ead7f8bc690e2dc94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:33:15.477804 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.475149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.21.41.193:38609)
0809 00:33:15.475194 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:15.475197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:15.475250 (+ 53us) server_negotiation.cc:407] Connection header received
0809 00:33:15.475327 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:15.475328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:15.475354 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:15.475434 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:33:15.475539 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.476262 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.477001 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.477306 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.477383 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.477496 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:33:15.477615 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:33:15.477694 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:15.477714 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 00:33:15.477736 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:33:15.482621 29722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.480091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:45015 (local address 127.21.41.194:41023)
0809 00:33:15.480137 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:15.480139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:15.480213 (+ 74us) server_negotiation.cc:407] Connection header received
0809 00:33:15.480296 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:15.480296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:15.480304 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:15.480347 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:33:15.480461 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.480889 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.481932 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.482103 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.482152 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.482398 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:33:15.482433 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:33:15.482470 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:15.482554 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 00:33:15.482571 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:33:15.482674 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.479987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:41023 (local address 127.21.41.193:45015)
0809 00:33:15.480153 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:15.480165 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:15.480229 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:15.480351 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:15.480352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:15.480353 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:33:15.480402 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.480403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.480970 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.480971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.481846 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.481849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.482182 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.482183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.482238 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.482255 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:33:15.482489 (+ 234us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:33:15.482492 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:15.482553 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:33:15.482595 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260809 00:33:15.492725 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.490064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:40921 (local address 127.21.41.193:38609)
0809 00:33:15.490118 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:15.490120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:15.490234 (+ 114us) server_negotiation.cc:407] Connection header received
0809 00:33:15.490292 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:15.490293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:15.490306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:15.490379 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:33:15.490526 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.491220 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.491993 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.492227 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.492299 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.492480 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:33:15.492513 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:33:15.492567 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:15.492625 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 00:33:15.492644 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:33:15.492760 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.490027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38609 (local address 127.21.41.194:40921)
0809 00:33:15.490167 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:15.490178 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:15.490246 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:15.490397 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:15.490397 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:15.490399 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:33:15.490461 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.490462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.491303 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.491304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.491895 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:15.491897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.492297 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.492298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:15.492366 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.492387 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:33:15.492570 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:33:15.492572 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:33:15.492621 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 00:33:15.492663 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":79,"threads_started":1}
I20260809 00:33:15.502060 29211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:15.502190 29211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:15.502925 29211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0: Sending DeleteTablet for 3 replicas of tablet 566bbdd2f2734fc0afe7104ba9ff7374
I20260809 00:33:15.503268 29619 tablet_service.cc:1514] Processing DeleteTablet for tablet 566bbdd2f2734fc0afe7104ba9ff7374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:15 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:46960
I20260809 00:33:15.503343 29349 tablet_service.cc:1514] Processing DeleteTablet for tablet 566bbdd2f2734fc0afe7104ba9ff7374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:15 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:60710
I20260809 00:33:15.503539 29489 tablet_service.cc:1514] Processing DeleteTablet for tablet 566bbdd2f2734fc0afe7104ba9ff7374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:15 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:50630
I20260809 00:33:15.503742 29796 tablet_replica.cc:330] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d: stopping tablet replica
I20260809 00:33:15.503845 29796 raft_consensus.cc:2229] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:15.503935 29796 raft_consensus.cc:2258] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:15.504040 29795 tablet_replica.cc:330] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663: stopping tablet replica
I20260809 00:33:15.504104 29795 raft_consensus.cc:2229] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:33:15.504199 29795 raft_consensus.cc:2258] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:15.504316 29794 tablet_replica.cc:330] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94: stopping tablet replica
I20260809 00:33:15.504377 29794 raft_consensus.cc:2229] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:15.504421 29794 raft_consensus.cc:2258] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:15.504496 29796 ts_tablet_manager.cc:1905] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:15.504680 29794 ts_tablet_manager.cc:1905] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:15.506567 29795 ts_tablet_manager.cc:1905] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:15.506814 29796 ts_tablet_manager.cc:1918] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:33:15.506919 29796 log.cc:1196] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/566bbdd2f2734fc0afe7104ba9ff7374
I20260809 00:33:15.507208 29796 ts_tablet_manager.cc:1939] T 566bbdd2f2734fc0afe7104ba9ff7374 P 8b0f73b49bf444bdb09e3cd15a4c611d: Deleting consensus metadata
I20260809 00:33:15.507584 29196 catalog_manager.cc:4841] TS 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023): tablet 566bbdd2f2734fc0afe7104ba9ff7374 (table test-table [id=f34bb968dc6245f0abb29667885d06a7]) successfully deleted
I20260809 00:33:15.508332 29795 ts_tablet_manager.cc:1918] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:33:15.508455 29795 log.cc:1196] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/wal/wals/566bbdd2f2734fc0afe7104ba9ff7374
I20260809 00:33:15.508517 29794 ts_tablet_manager.cc:1918] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:33:15.508582 29794 log.cc:1196] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/566bbdd2f2734fc0afe7104ba9ff7374
I20260809 00:33:15.508823 29795 ts_tablet_manager.cc:1939] T 566bbdd2f2734fc0afe7104ba9ff7374 P d78f2dd2181747a3b537106a7b59b663: Deleting consensus metadata
I20260809 00:33:15.508836 29794 ts_tablet_manager.cc:1939] T 566bbdd2f2734fc0afe7104ba9ff7374 P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting consensus metadata
I20260809 00:33:15.509214 29198 catalog_manager.cc:4841] TS d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609): tablet 566bbdd2f2734fc0afe7104ba9ff7374 (table test-table [id=f34bb968dc6245f0abb29667885d06a7]) successfully deleted
I20260809 00:33:15.509518 29196 catalog_manager.cc:4841] TS bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195:37459): tablet 566bbdd2f2734fc0afe7104ba9ff7374 (table test-table [id=f34bb968dc6245f0abb29667885d06a7]) successfully deleted
Aug 09 00:33:15 dist-test-slave-zpfg krb5kdc[29167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:33:15 dist-test-slave-zpfg krb5kdc[29167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:15 dist-test-slave-zpfg krb5kdc[29167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 00:33:15.542371 29804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:15.534932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36570 (local address 127.21.41.254:46733)
0809 00:33:15.535039 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:15.535041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:15.535130 (+ 89us) server_negotiation.cc:407] Connection header received
0809 00:33:15.535198 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:15.535199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:15.535218 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:15.535302 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:15.536048 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.536906 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.538085 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.538432 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.538532 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.542165 (+ 3633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36570: BlockingRecv error: recv got EOF from 127.0.0.1:36570 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":51,"threads_started":1}
Aug 09 00:33:15 dist-test-slave-zpfg krb5kdc[29167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM
I20260809 00:33:15.551849 29804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.543057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36576 (local address 127.21.41.254:46733)
0809 00:33:15.543092 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:15.543093 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:15.543332 (+ 239us) server_negotiation.cc:407] Connection header received
0809 00:33:15.543381 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:15.543382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:15.543393 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:15.543453 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:15.544029 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.544799 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.545836 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.546150 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.546229 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.548955 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:15.548956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:15.548956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:15.548964 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:15.550673 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:15.551050 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:15.551051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:15.551052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:15.551112 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:15.551318 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:15.551319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:15.551319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:15.551448 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:15.551543 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:15.551669 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 00:33:15.551747 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:33:15.553715 29211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:33:15.553900 29211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:33:15.555402 29489 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccfa4a297454d68bb4a1768cd76f928 (DEFAULT_TABLE table=test-table [id=b66801bba7414f0e957c30b31aeb9de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:15.555498 29349 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccfa4a297454d68bb4a1768cd76f928 (DEFAULT_TABLE table=test-table [id=b66801bba7414f0e957c30b31aeb9de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:15.555545 29489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccfa4a297454d68bb4a1768cd76f928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:15.555640 29349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccfa4a297454d68bb4a1768cd76f928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:15.557047 29619 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccfa4a297454d68bb4a1768cd76f928 (DEFAULT_TABLE table=test-table [id=b66801bba7414f0e957c30b31aeb9de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:15.557165 29619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccfa4a297454d68bb4a1768cd76f928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:15.557842 29810 tablet_bootstrap.cc:492] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663: Bootstrap starting.
I20260809 00:33:15.558302 29810 tablet_bootstrap.cc:654] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:15.558782 29811 tablet_bootstrap.cc:492] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d: Bootstrap starting.
I20260809 00:33:15.559072 29810 tablet_bootstrap.cc:492] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663: No bootstrap required, opened a new log
I20260809 00:33:15.559146 29810 ts_tablet_manager.cc:1397] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:15.559154 29811 tablet_bootstrap.cc:654] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:15.559320 29810 raft_consensus.cc:348] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.559417 29810 raft_consensus.cc:374] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:15.559432 29813 tablet_bootstrap.cc:492] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94: Bootstrap starting.
I20260809 00:33:15.559458 29810 raft_consensus.cc:729] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78f2dd2181747a3b537106a7b59b663, State: Initialized, Role: FOLLOWER
I20260809 00:33:15.559578 29810 consensus_queue.cc:260] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.559717 29810 ts_tablet_manager.cc:1428] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:33:15.559883 29813 tablet_bootstrap.cc:654] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:15.560168 29811 tablet_bootstrap.cc:492] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d: No bootstrap required, opened a new log
I20260809 00:33:15.560221 29811 ts_tablet_manager.cc:1397] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:15.560369 29811 raft_consensus.cc:348] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.560448 29811 raft_consensus.cc:374] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:15.560469 29811 raft_consensus.cc:729] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b0f73b49bf444bdb09e3cd15a4c611d, State: Initialized, Role: FOLLOWER
I20260809 00:33:15.560524 29811 consensus_queue.cc:260] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.560631 29811 ts_tablet_manager.cc:1428] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:15.561336 29813 tablet_bootstrap.cc:492] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94: No bootstrap required, opened a new log
I20260809 00:33:15.561394 29813 ts_tablet_manager.cc:1397] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:33:15.561514 29813 raft_consensus.cc:348] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.561583 29813 raft_consensus.cc:374] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:15.561606 29813 raft_consensus.cc:729] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc98ba70cd914f7ead7f8bc690e2dc94, State: Initialized, Role: FOLLOWER
I20260809 00:33:15.561656 29813 consensus_queue.cc:260] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.561751 29813 ts_tablet_manager.cc:1428] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:15.621789 29743 raft_consensus.cc:482] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:33:15.621904 29743 raft_consensus.cc:504] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.622071 29743 leader_election.cc:288] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609), 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023)
I20260809 00:33:15.622366 29369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccfa4a297454d68bb4a1768cd76f928" candidate_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78f2dd2181747a3b537106a7b59b663" is_pre_election: true
I20260809 00:33:15.622470 29369 raft_consensus.cc:2454] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc98ba70cd914f7ead7f8bc690e2dc94 in term 0.
I20260809 00:33:15.622630 29582 leader_election.cc:302] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc98ba70cd914f7ead7f8bc690e2dc94, d78f2dd2181747a3b537106a7b59b663; no voters:
I20260809 00:33:15.622318 29509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccfa4a297454d68bb4a1768cd76f928" candidate_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" is_pre_election: true
I20260809 00:33:15.623059 29509 raft_consensus.cc:2454] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc98ba70cd914f7ead7f8bc690e2dc94 in term 0.
I20260809 00:33:15.623919 29743 raft_consensus.cc:2781] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:33:15.623968 29743 raft_consensus.cc:482] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:33:15.623989 29743 raft_consensus.cc:3037] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:15.624924 29743 raft_consensus.cc:504] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.625049 29743 leader_election.cc:288] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [CANDIDATE]: Term 1 election: Requested vote from peers d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609), 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023)
I20260809 00:33:15.625236 29369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccfa4a297454d68bb4a1768cd76f928" candidate_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78f2dd2181747a3b537106a7b59b663"
I20260809 00:33:15.625304 29369 raft_consensus.cc:3037] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:15.626009 29369 raft_consensus.cc:2454] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc98ba70cd914f7ead7f8bc690e2dc94 in term 1.
I20260809 00:33:15.626165 29582 leader_election.cc:302] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc98ba70cd914f7ead7f8bc690e2dc94, d78f2dd2181747a3b537106a7b59b663; no voters:
I20260809 00:33:15.626245 29743 raft_consensus.cc:2781] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:15.626374 29743 raft_consensus.cc:686] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 LEADER]: Becoming Leader. State: Replica: bc98ba70cd914f7ead7f8bc690e2dc94, State: Running, Role: LEADER
I20260809 00:33:15.626422 29743 consensus_queue.cc:237] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.627020 29509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccfa4a297454d68bb4a1768cd76f928" candidate_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d"
I20260809 00:33:15.626984 29211 catalog_manager.cc:5495] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195). New cstate: current_term: 1 leader_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } health_report { overall_health: HEALTHY } } }
I20260809 00:33:15.627115 29509 raft_consensus.cc:3037] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:15.627743 29509 raft_consensus.cc:2454] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc98ba70cd914f7ead7f8bc690e2dc94 in term 1.
I20260809 00:33:15.635236 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.632831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46968 (local address 127.21.41.195:37459)
0809 00:33:15.632913 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:15.632915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:15.632924 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:15.632972 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:15.632973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:15.632983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:15.633057 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:33:15.633206 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.633680 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.634533 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.634730 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.634817 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.634972 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:33:15.635010 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:33:15.635072 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:15.635132 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 00:33:15.635153 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 00:33:15.635989 29509 raft_consensus.cc:1261] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Refusing update from remote peer bc98ba70cd914f7ead7f8bc690e2dc94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:15.636022 29369 raft_consensus.cc:1261] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Refusing update from remote peer bc98ba70cd914f7ead7f8bc690e2dc94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:15.636236 29743 consensus_queue.cc:1035] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:15.636324 29816 consensus_queue.cc:1035] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:15.638990 29212 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:15.639106 29212 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:15.639794 29212 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0: Sending DeleteTablet for 3 replicas of tablet 1ccfa4a297454d68bb4a1768cd76f928
I20260809 00:33:15.640055 29348 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ccfa4a297454d68bb4a1768cd76f928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:15 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:60710
I20260809 00:33:15.640110 29619 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ccfa4a297454d68bb4a1768cd76f928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:15 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:46960
I20260809 00:33:15.640105 29489 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ccfa4a297454d68bb4a1768cd76f928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:15 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:50630
I20260809 00:33:15.640164 29795 tablet_replica.cc:330] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663: stopping tablet replica
I20260809 00:33:15.640215 29795 raft_consensus.cc:2229] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:15.640234 29794 tablet_replica.cc:330] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94: stopping tablet replica
I20260809 00:33:15.640259 29795 raft_consensus.cc:2258] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:15.640287 29794 raft_consensus.cc:2229] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:33:15.640337 29794 raft_consensus.cc:2258] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:15.640396 29796 tablet_replica.cc:330] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d: stopping tablet replica
I20260809 00:33:15.640455 29796 raft_consensus.cc:2229] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:15.640497 29796 raft_consensus.cc:2258] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:15.640072 29349 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ccfa4a297454d68bb4a1768cd76f928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:15 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:60710
I20260809 00:33:15.640692 29212 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:33:15.640843 29212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:33:15.640895 29794 ts_tablet_manager.cc:1905] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:15.640947 29795 ts_tablet_manager.cc:1905] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:15.642493 29795 ts_tablet_manager.cc:1918] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:15.642542 29795 log.cc:1196] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-0/wal/wals/1ccfa4a297454d68bb4a1768cd76f928
I20260809 00:33:15.642735 29795 ts_tablet_manager.cc:1939] T 1ccfa4a297454d68bb4a1768cd76f928 P d78f2dd2181747a3b537106a7b59b663: Deleting consensus metadata
I20260809 00:33:15.643118 29794 ts_tablet_manager.cc:1918] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:15.643182 29794 log.cc:1196] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/1ccfa4a297454d68bb4a1768cd76f928
I20260809 00:33:15.643344 29349 tablet_service.cc:1467] Processing CreateTablet for tablet 6b245eef80f547c7b4b99f684c2256fd (DEFAULT_TABLE table=test-table [id=6c90679bc9164320967a92c553547f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:15.643399 29794 ts_tablet_manager.cc:1939] T 1ccfa4a297454d68bb4a1768cd76f928 P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting consensus metadata
I20260809 00:33:15.643450 29349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b245eef80f547c7b4b99f684c2256fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:15.643500 29489 tablet_service.cc:1467] Processing CreateTablet for tablet 6b245eef80f547c7b4b99f684c2256fd (DEFAULT_TABLE table=test-table [id=6c90679bc9164320967a92c553547f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:15.643603 29489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b245eef80f547c7b4b99f684c2256fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:15.644488 29810 tablet_bootstrap.cc:492] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663: Bootstrap starting.
I20260809 00:33:15.644793 29810 tablet_bootstrap.cc:654] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:15.644992 29796 ts_tablet_manager.cc:1905] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:15.645053 29811 tablet_bootstrap.cc:492] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d: Bootstrap starting.
I20260809 00:33:15.645364 29811 tablet_bootstrap.cc:654] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:15.645541 29619 tablet_service.cc:1467] Processing CreateTablet for tablet 6b245eef80f547c7b4b99f684c2256fd (DEFAULT_TABLE table=test-table [id=6c90679bc9164320967a92c553547f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:15.645649 29619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b245eef80f547c7b4b99f684c2256fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:15.645898 29811 tablet_bootstrap.cc:492] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d: No bootstrap required, opened a new log
I20260809 00:33:15.645974 29811 ts_tablet_manager.cc:1397] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:15.646090 29811 raft_consensus.cc:348] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.646184 29811 raft_consensus.cc:374] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:15.646212 29811 raft_consensus.cc:729] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b0f73b49bf444bdb09e3cd15a4c611d, State: Initialized, Role: FOLLOWER
I20260809 00:33:15.646255 29811 consensus_queue.cc:260] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.646397 29811 ts_tablet_manager.cc:1428] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:15.646485 29198 catalog_manager.cc:4841] TS d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609): tablet 1ccfa4a297454d68bb4a1768cd76f928 (table test-table [id=b66801bba7414f0e957c30b31aeb9de3]) successfully deleted
W20260809 00:33:15.646620 29198 catalog_manager.cc:4808] TS d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609): delete failed for tablet 1ccfa4a297454d68bb4a1768cd76f928 because the tablet was not found. No further retry: Not found: Tablet not found: 1ccfa4a297454d68bb4a1768cd76f928
I20260809 00:33:15.646888 29796 ts_tablet_manager.cc:1918] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:15.646946 29796 log.cc:1196] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/1ccfa4a297454d68bb4a1768cd76f928
I20260809 00:33:15.646955 29813 tablet_bootstrap.cc:492] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94: Bootstrap starting.
I20260809 00:33:15.647012 29196 catalog_manager.cc:4841] TS bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195:37459): tablet 1ccfa4a297454d68bb4a1768cd76f928 (table test-table [id=b66801bba7414f0e957c30b31aeb9de3]) successfully deleted
I20260809 00:33:15.647168 29796 ts_tablet_manager.cc:1939] T 1ccfa4a297454d68bb4a1768cd76f928 P 8b0f73b49bf444bdb09e3cd15a4c611d: Deleting consensus metadata
I20260809 00:33:15.647331 29813 tablet_bootstrap.cc:654] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:15.647449 29810 tablet_bootstrap.cc:492] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663: No bootstrap required, opened a new log
I20260809 00:33:15.647501 29810 ts_tablet_manager.cc:1397] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 00:33:15.647507 29196 catalog_manager.cc:4841] TS 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023): tablet 1ccfa4a297454d68bb4a1768cd76f928 (table test-table [id=b66801bba7414f0e957c30b31aeb9de3]) successfully deleted
I20260809 00:33:15.647619 29810 raft_consensus.cc:348] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.647704 29810 raft_consensus.cc:374] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:15.647737 29810 raft_consensus.cc:729] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78f2dd2181747a3b537106a7b59b663, State: Initialized, Role: FOLLOWER
I20260809 00:33:15.647791 29810 consensus_queue.cc:260] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.647890 29810 ts_tablet_manager.cc:1428] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:15.648022 29813 tablet_bootstrap.cc:492] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94: No bootstrap required, opened a new log
I20260809 00:33:15.648072 29813 ts_tablet_manager.cc:1397] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:15.648171 29813 raft_consensus.cc:348] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.648250 29813 raft_consensus.cc:374] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:15.648288 29813 raft_consensus.cc:729] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc98ba70cd914f7ead7f8bc690e2dc94, State: Initialized, Role: FOLLOWER
I20260809 00:33:15.648339 29813 consensus_queue.cc:260] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.648444 29813 ts_tablet_manager.cc:1428] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:15.746454 29732 raft_consensus.cc:482] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:33:15.746570 29732 raft_consensus.cc:504] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.746776 29732 leader_election.cc:288] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023), bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195:37459)
I20260809 00:33:15.747068 29509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b245eef80f547c7b4b99f684c2256fd" candidate_uuid: "d78f2dd2181747a3b537106a7b59b663" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" is_pre_election: true
I20260809 00:33:15.747102 29638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b245eef80f547c7b4b99f684c2256fd" candidate_uuid: "d78f2dd2181747a3b537106a7b59b663" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" is_pre_election: true
I20260809 00:33:15.747179 29509 raft_consensus.cc:2454] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d78f2dd2181747a3b537106a7b59b663 in term 0.
I20260809 00:33:15.747200 29638 raft_consensus.cc:2454] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d78f2dd2181747a3b537106a7b59b663 in term 0.
I20260809 00:33:15.747375 29300 leader_election.cc:302] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b0f73b49bf444bdb09e3cd15a4c611d, d78f2dd2181747a3b537106a7b59b663; no voters:
I20260809 00:33:15.747493 29732 raft_consensus.cc:2781] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:33:15.747545 29732 raft_consensus.cc:482] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:33:15.747571 29732 raft_consensus.cc:3037] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:15.748319 29732 raft_consensus.cc:504] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.748451 29732 leader_election.cc:288] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [CANDIDATE]: Term 1 election: Requested vote from peers 8b0f73b49bf444bdb09e3cd15a4c611d (127.21.41.194:41023), bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195:37459)
I20260809 00:33:15.748622 29509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b245eef80f547c7b4b99f684c2256fd" candidate_uuid: "d78f2dd2181747a3b537106a7b59b663" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d"
I20260809 00:33:15.748703 29509 raft_consensus.cc:3037] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:15.748652 29638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b245eef80f547c7b4b99f684c2256fd" candidate_uuid: "d78f2dd2181747a3b537106a7b59b663" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94"
I20260809 00:33:15.748749 29638 raft_consensus.cc:3037] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:15.749330 29509 raft_consensus.cc:2454] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d78f2dd2181747a3b537106a7b59b663 in term 1.
I20260809 00:33:15.749452 29638 raft_consensus.cc:2454] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d78f2dd2181747a3b537106a7b59b663 in term 1.
I20260809 00:33:15.749531 29300 leader_election.cc:302] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b0f73b49bf444bdb09e3cd15a4c611d, d78f2dd2181747a3b537106a7b59b663; no voters:
I20260809 00:33:15.749610 29732 raft_consensus.cc:2781] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:15.749675 29732 raft_consensus.cc:686] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 1 LEADER]: Becoming Leader. State: Replica: d78f2dd2181747a3b537106a7b59b663, State: Running, Role: LEADER
I20260809 00:33:15.749737 29732 consensus_queue.cc:237] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } }
I20260809 00:33:15.750331 29211 catalog_manager.cc:5495] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 reported cstate change: term changed from 0 to 1, leader changed from <none> to d78f2dd2181747a3b537106a7b59b663 (127.21.41.193). New cstate: current_term: 1 leader_uuid: "d78f2dd2181747a3b537106a7b59b663" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78f2dd2181747a3b537106a7b59b663" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 38609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 } health_report { overall_health: UNKNOWN } } }
I20260809 00:33:15.766053 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.763766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.21.41.193:38609)
0809 00:33:15.763824 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:15.763827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:15.763868 (+ 41us) server_negotiation.cc:407] Connection header received
0809 00:33:15.763945 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:15.763946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:15.763955 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:15.764023 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:33:15.764149 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.764623 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.765362 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:15.765559 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:15.765636 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:15.765790 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:33:15.765828 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:33:15.765889 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:15.765951 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 00:33:15.765971 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:33:15.768153 29638 raft_consensus.cc:1261] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Refusing update from remote peer d78f2dd2181747a3b537106a7b59b663: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:15.768185 29509 raft_consensus.cc:1261] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Refusing update from remote peer d78f2dd2181747a3b537106a7b59b663: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:15.768385 29732 consensus_queue.cc:1035] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc98ba70cd914f7ead7f8bc690e2dc94" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:15.768533 29754 consensus_queue.cc:1035] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b0f73b49bf444bdb09e3cd15a4c611d" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 41023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:15.778326 29212 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:15.778438 29212 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:15.779102 29212 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 165cc746ca43445d9a0aa0f9a4c1a6a0: Sending DeleteTablet for 3 replicas of tablet 6b245eef80f547c7b4b99f684c2256fd
I20260809 00:33:15.779335 29349 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b245eef80f547c7b4b99f684c2256fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:15 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:60710
I20260809 00:33:15.779439 29489 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b245eef80f547c7b4b99f684c2256fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:15 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:50630
I20260809 00:33:15.779445 29795 tablet_replica.cc:330] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663: stopping tablet replica
I20260809 00:33:15.779485 29619 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b245eef80f547c7b4b99f684c2256fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:15 UTC) from {username='oryx', principal='oryx/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:46960
I20260809 00:33:15.779520 29795 raft_consensus.cc:2229] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:33:15.779536 29796 tablet_replica.cc:330] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d: stopping tablet replica
I20260809 00:33:15.779572 29795 raft_consensus.cc:2258] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:15.779570 29794 tablet_replica.cc:330] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94: stopping tablet replica
I20260809 00:33:15.779592 29796 raft_consensus.cc:2229] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:15.779623 29794 raft_consensus.cc:2229] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:15.779636 29796 raft_consensus.cc:2258] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:15.779659 29794 raft_consensus.cc:2258] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:15.779862 29795 ts_tablet_manager.cc:1905] T 6b245eef80f547c7b4b99f684c2256fd P d78f2dd2181747a3b537106a7b59b663: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:15.779917 29794 ts_tablet_manager.cc:1905] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:15.779938 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 29284
I20260809 00:33:15.781659 29794 ts_tablet_manager.cc:1918] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:33:15.781713 29794 log.cc:1196] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/6b245eef80f547c7b4b99f684c2256fd
I20260809 00:33:15.781935 29794 ts_tablet_manager.cc:1939] T 6b245eef80f547c7b4b99f684c2256fd P bc98ba70cd914f7ead7f8bc690e2dc94: Deleting consensus metadata
I20260809 00:33:15.785678 29196 catalog_manager.cc:4841] TS bc98ba70cd914f7ead7f8bc690e2dc94 (127.21.41.195:37459): tablet 6b245eef80f547c7b4b99f684c2256fd (table test-table [id=6c90679bc9164320967a92c553547f27]) successfully deleted
I20260809 00:33:15.786651 29796 ts_tablet_manager.cc:1905] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:33:15.787632 29198 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:38609 (error 108)
I20260809 00:33:15.787743 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 29424
I20260809 00:33:15.788163 29796 ts_tablet_manager.cc:1918] T 6b245eef80f547c7b4b99f684c2256fd P 8b0f73b49bf444bdb09e3cd15a4c611d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:33:15.789357 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.788596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:38609 (local address 127.0.0.1:60736)
0809 00:33:15.789282 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:15.789323 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:38609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":39,"threads_started":1}
W20260809 00:33:15.789480 29198 catalog_manager.cc:4584] TS d78f2dd2181747a3b537106a7b59b663 (127.21.41.193:38609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b245eef80f547c7b4b99f684c2256fd: Network error: Client connection negotiation failed: client connection to 127.21.41.193:38609: connect: Connection refused (error 111)
I20260809 00:33:15.794071 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:15.793961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:41023 (local address 127.0.0.1:50660)
0809 00:33:15.794013 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:15.794044 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.194:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:33:15.794090 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 29564
I20260809 00:33:15.800057 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 29180
2026-08-09T00:33:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2880 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:33:15 dist-test-slave-zpfg krb5kdc[29832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:33:15 dist-test-slave-zpfg krb5kdc[29832](info): set up 2 sockets
Aug 09 00:33:15 dist-test-slave-zpfg krb5kdc[29832](info): commencing operation
krb5kdc: starting...
W20260809 00:33:16.840675 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:33:16 dist-test-slave-zpfg krb5kdc[29832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:33:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:33:16.965579 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:33701
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:33701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:17.048336 29845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:17.048597 29845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:17.048636 29845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:17.049788 29845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:17.049861 29845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:17.049886 29845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:17.049908 29845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:17.049938 29845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:17.051472 29845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:33701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:33701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:17.051765 29845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:17.052001 29845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:17.057053 29853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:17.057138 29850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:17.057035 29851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:17.057336 29845 server_base.cc:1034] running on GCE node
I20260809 00:33:17.057581 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:17.057873 29845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:17.059046 29845 hybrid_clock.cc:648] HybridClock initialized: now 1786235597059004 us; error 58 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T00:33:17Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1261 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:33:17 dist-test-slave-zpfg krb5kdc[29858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:33:17 dist-test-slave-zpfg krb5kdc[29858](info): set up 2 sockets
Aug 09 00:33:17 dist-test-slave-zpfg krb5kdc[29858](info): commencing operation
krb5kdc: starting...
W20260809 00:33:18.104718 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:33:18 dist-test-slave-zpfg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:33:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:33:18.237486 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:46763
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:46763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:18.319382 29871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:18.319640 29871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:18.319679 29871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:18.320818 29871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:18.320894 29871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:18.320919 29871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:18.320948 29871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:18.320979 29871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:18.322489 29871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:46763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:46763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:18.322774 29871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:18.323001 29871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:18.328214 29877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:18.328190 29879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:18.328255 29871 server_base.cc:1034] running on GCE node
W20260809 00:33:18.328195 29876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:18.328739 29871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:18.329080 29871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:18.330262 29871 hybrid_clock.cc:648] HybridClock initialized: now 1786235598330255 us; error 38 us; skew 500 ppm
Aug 09 00:33:18 dist-test-slave-zpfg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:18.332898 29871 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:33:18.333352 29871 webserver.cc:460] Webserver started at http://127.21.41.254:46615/ using document root <none> and password file <none>
I20260809 00:33:18.333555 29871 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:18.333604 29871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:18.333691 29871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:18.334758 29871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "46fb58b4f9a04be9bbef5f548769101e"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "17c8cc16901dd05d5c032f5953b8bdcd"
server_key_iv: "4bd4d2eb8af13576fc2750f5e91f4b24"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.335053 29871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "46fb58b4f9a04be9bbef5f548769101e"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "17c8cc16901dd05d5c032f5953b8bdcd"
server_key_iv: "4bd4d2eb8af13576fc2750f5e91f4b24"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.336377 29871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:33:18.337317 29886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:18.337540 29871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:18.337647 29871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "46fb58b4f9a04be9bbef5f548769101e"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "17c8cc16901dd05d5c032f5953b8bdcd"
server_key_iv: "4bd4d2eb8af13576fc2750f5e91f4b24"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.337702 29871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:18.356411 29871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:18.358587 29871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:18.358701 29871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:18.362540 29871 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:46763
I20260809 00:33:18.362552 29938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:46763 every 8 connection(s)
I20260809 00:33:18.362916 29871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:18.363786 29939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:18.366070 29939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e: Bootstrap starting.
I20260809 00:33:18.366700 29939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:18.366998 29939 log.cc:824] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:18.367816 29939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e: No bootstrap required, opened a new log
I20260809 00:33:18.369413 29939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } }
I20260809 00:33:18.369551 29939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:18.369575 29939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46fb58b4f9a04be9bbef5f548769101e, State: Initialized, Role: FOLLOWER
I20260809 00:33:18.369768 29939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } }
I20260809 00:33:18.369855 29939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:18.369908 29939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:18.369958 29939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:18.370553 29939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } }
I20260809 00:33:18.370668 29939 leader_election.cc:302] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46fb58b4f9a04be9bbef5f548769101e; no voters:
I20260809 00:33:18.370867 29939 leader_election.cc:288] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:18.370946 29942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:18.371124 29942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 1 LEADER]: Becoming Leader. State: Replica: 46fb58b4f9a04be9bbef5f548769101e, State: Running, Role: LEADER
I20260809 00:33:18.371145 29939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:18.371212 29942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } }
I20260809 00:33:18.371587 29943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46fb58b4f9a04be9bbef5f548769101e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } } }
I20260809 00:33:18.371686 29943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:18.371592 29944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46fb58b4f9a04be9bbef5f548769101e. Latest consensus state: current_term: 1 leader_uuid: "46fb58b4f9a04be9bbef5f548769101e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } } }
I20260809 00:33:18.371846 29944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:18.372020 29946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:18.372505 29946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:18.372841 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 29871
I20260809 00:33:18.372951 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:18.373144 21671 external_mini_cluster.cc:1428] Setting key 3de2e63cba37fa7776290573799297e7
I20260809 00:33:18.374398 29946 catalog_manager.cc:1347] Generated new cluster ID: 38642032d3c8477a972098a3679d3f49
I20260809 00:33:18.374452 29946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:33:18 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:18.385789 29961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:18.373688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.21.41.254:46763)
0809 00:33:18.373878 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:18.373888 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:18.373973 (+ 85us) server_negotiation.cc:407] Connection header received
0809 00:33:18.374145 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:18.374153 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:18.374211 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:18.375183 (+ 972us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:18.375276 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.376727 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.377764 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.378428 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.378542 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:18.381358 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:18.381363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:18.381363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:18.381377 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:18.383575 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:18.384041 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:18.384044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:18.384044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:18.384169 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:18.384389 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:18.384390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:18.384390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:18.384727 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:18.385313 (+ 586us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:18.385385 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 00:33:18.385524 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260809 00:33:18.391402 29946 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:18.391976 29946 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:18.394863 29946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e: Generated new TSK 0
I20260809 00:33:18.395042 29946 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:33:18.421505 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46763
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:18.504628 29965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:18.504843 29965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:18.504866 29965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:18.506111 29965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:18.506150 29965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:18.506178 29965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:33:18.507656 29965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:18.507910 29965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:18.508150 29965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:18.513377 29971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:18.513358 29973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:18.513549 29970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:18.513689 29965 server_base.cc:1034] running on GCE node
I20260809 00:33:18.513952 29965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:18.514178 29965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:18.515321 29965 hybrid_clock.cc:648] HybridClock initialized: now 1786235598515307 us; error 31 us; skew 500 ppm
Aug 09 00:33:18 dist-test-slave-zpfg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:18.517843 29965 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:33:18.518309 29965 webserver.cc:460] Webserver started at http://127.21.41.193:43843/ using document root <none> and password file <none>
I20260809 00:33:18.518558 29965 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:18.518604 29965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:18.518693 29965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:18.519663 29965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "e5a4fab2a16948119c4f71fc896b4f8b"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "2ffe06786fa3f949c4277e41f732f377"
server_key_iv: "50a12105d73534f8e5107e506d40174c"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.519970 29965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "e5a4fab2a16948119c4f71fc896b4f8b"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "2ffe06786fa3f949c4277e41f732f377"
server_key_iv: "50a12105d73534f8e5107e506d40174c"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.521169 29965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:33:18.521979 29980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:18.522212 29965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:18.522312 29965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "e5a4fab2a16948119c4f71fc896b4f8b"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "2ffe06786fa3f949c4277e41f732f377"
server_key_iv: "50a12105d73534f8e5107e506d40174c"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.522441 29965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:18.543036 29965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:18.545177 29965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:18.545353 29965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:18.545598 29965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:18.545935 29965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:18.545998 29965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:18.546041 29965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:18.546069 29965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:18.552789 29965 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:35855
I20260809 00:33:18.552875 30093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:35855 every 8 connection(s)
I20260809 00:33:18.553205 29965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:33:18.556555 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 29965
I20260809 00:33:18.556658 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:33:18.556866 21671 external_mini_cluster.cc:1428] Setting key 05d42c524589d363ee0d546bdd18d95d
Aug 09 00:33:18 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:18.563534 29961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:18.553946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:56223 (local address 127.21.41.254:46763)
0809 00:33:18.553993 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:18.553995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:18.554216 (+ 221us) server_negotiation.cc:407] Connection header received
0809 00:33:18.554816 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:18.554817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:18.554827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:18.554888 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:18.555914 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.556483 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.557838 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.558154 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.558232 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:18.560522 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:18.560524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:18.560525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:18.560532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:18.561965 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:18.562512 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:18.562513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:18.562514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:18.562569 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:18.562868 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:18.562868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:18.562869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:18.562975 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:18.563025 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:18.563392 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 00:33:18.563466 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:33:18.563807 30096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:18.553924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:56223)
0809 00:33:18.554097 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:18.554141 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:18.554700 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:18.554969 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:18.554972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:18.555574 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:18.555828 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:18.555833 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.556585 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.556586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:18.557732 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:18.557734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.558263 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.558264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:18.558449 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:18.559017 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:18.559026 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:18.560417 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:18.562068 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:18.562076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:18.562079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:18.562433 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:18.562644 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:18.562645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:18.562645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:18.562794 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:18.563041 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:18.563049 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:18.563242 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 00:33:18.563364 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 00:33:18.563549 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260809 00:33:18.564378 30094 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46763
I20260809 00:33:18.564477 30094 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:18.564677 30094 heartbeater.cc:511] Master 127.21.41.254:46763 requested a full tablet report, sending...
I20260809 00:33:18.565181 29903 ts_manager.cc:194] Registered new tserver with Master: e5a4fab2a16948119c4f71fc896b4f8b (127.21.41.193:35855)
I20260809 00:33:18.565810 29903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:56223
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:33:18.585331 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46763
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:18.668237 30099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:18.668514 30099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:18.668552 30099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:18.669946 30099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:18.670023 30099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:18.670073 30099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:33:18.671689 30099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:18.671939 30099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:18.672179 30099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:18.677309 30105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:18.677323 30107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:18.677309 30104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:18.677570 30099 server_base.cc:1034] running on GCE node
I20260809 00:33:18.677806 30099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:18.678045 30099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:18.679183 30099 hybrid_clock.cc:648] HybridClock initialized: now 1786235598679171 us; error 30 us; skew 500 ppm
Aug 09 00:33:18 dist-test-slave-zpfg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:18.681569 30099 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:33:18.682020 30099 webserver.cc:460] Webserver started at http://127.21.41.194:38229/ using document root <none> and password file <none>
I20260809 00:33:18.682237 30099 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:18.682286 30099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:18.682484 30099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:18.683568 30099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "ed19688e5875496f97868cf69e34bb9f"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "67855b3250b8b0214d9e1424f7b3fdf1"
server_key_iv: "031d43c4c21b987780787161702e0284"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.683936 30099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "ed19688e5875496f97868cf69e34bb9f"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "67855b3250b8b0214d9e1424f7b3fdf1"
server_key_iv: "031d43c4c21b987780787161702e0284"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.685240 30099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:33:18.685992 30114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:18.686177 30099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:18.686278 30099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "ed19688e5875496f97868cf69e34bb9f"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "67855b3250b8b0214d9e1424f7b3fdf1"
server_key_iv: "031d43c4c21b987780787161702e0284"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.686383 30099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:18.703922 30099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:18.706010 30099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:18.706182 30099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:18.706462 30099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:18.706807 30099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:18.706864 30099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:18.706908 30099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:18.706933 30099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:18.712738 30099 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:46071
I20260809 00:33:18.712792 30227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:46071 every 8 connection(s)
I20260809 00:33:18.713161 30099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:33:18 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:18.720643 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 30099
I20260809 00:33:18.720788 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:33:18.721012 21671 external_mini_cluster.cc:1428] Setting key 4daf71187a929a0b67b43e0edd99d7db
I20260809 00:33:18.723130 29961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:18.713953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:35685 (local address 127.21.41.254:46763)
0809 00:33:18.714006 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:18.714026 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:18.714225 (+ 199us) server_negotiation.cc:407] Connection header received
0809 00:33:18.714786 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:18.714788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:18.714801 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:18.714866 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:18.715910 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.716417 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.717446 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.717640 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.717715 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:18.720011 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:18.720012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:18.720012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:18.720019 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:18.721539 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:18.722080 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:18.722081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:18.722082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:18.722151 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:18.722493 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:18.722494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:18.722494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:18.722608 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:18.722666 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:18.723001 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 00:33:18.723066 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:33:18.723444 30230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:18.713902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:35685)
0809 00:33:18.714089 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:18.714136 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:18.714658 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:18.714940 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:18.714945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:18.715553 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:18.715797 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:18.715802 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.716518 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.716519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:18.717337 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:18.717339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.717719 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.717720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:18.717867 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:18.718426 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:18.718435 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:18.719922 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:18.721637 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:18.721644 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:18.721646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:18.721978 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:18.722255 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:18.722255 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:18.722256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:18.722430 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:18.722679 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:18.722685 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:18.722857 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 00:33:18.722965 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 00:33:18.723145 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
I20260809 00:33:18.724140 30228 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46763
I20260809 00:33:18.724259 30228 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:18.724478 30228 heartbeater.cc:511] Master 127.21.41.254:46763 requested a full tablet report, sending...
I20260809 00:33:18.724896 29903 ts_manager.cc:194] Registered new tserver with Master: ed19688e5875496f97868cf69e34bb9f (127.21.41.194:46071)
I20260809 00:33:18.725428 29903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:35685
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:33:18.749810 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46763
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:18.833406 30233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:18.833618 30233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:18.833642 30233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:18.834888 30233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:18.834929 30233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:18.834956 30233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:18.836426 30233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:18.836679 30233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:18.836915 30233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:18.842023 30238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:18.841990 30239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:18.842000 30241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:18.842254 30233 server_base.cc:1034] running on GCE node
I20260809 00:33:18.842482 30233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:18.842681 30233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:18.843844 30233 hybrid_clock.cc:648] HybridClock initialized: now 1786235598843832 us; error 45 us; skew 500 ppm
Aug 09 00:33:18 dist-test-slave-zpfg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:18.846658 30233 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:33:18.847155 30233 webserver.cc:460] Webserver started at http://127.21.41.195:41925/ using document root <none> and password file <none>
I20260809 00:33:18.847406 30233 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:18.847474 30233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:18.847633 30233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:18.848773 30233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "38a949af19344a8da2684e7329875660"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "7710afb2a1da8f0ee390e933eecddfcd"
server_key_iv: "45edd7c17df8d42f17010c911a823a5b"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.849150 30233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "38a949af19344a8da2684e7329875660"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "7710afb2a1da8f0ee390e933eecddfcd"
server_key_iv: "45edd7c17df8d42f17010c911a823a5b"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.850656 30233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:33:18.851488 30248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:18.851713 30233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:33:18.851827 30233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "38a949af19344a8da2684e7329875660"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "7710afb2a1da8f0ee390e933eecddfcd"
server_key_iv: "45edd7c17df8d42f17010c911a823a5b"
server_key_version: "encryptionkey@0"
I20260809 00:33:18.851904 30233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:18.859807 30233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:18.861756 30233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:18.861924 30233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:18.862159 30233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:18.862548 30233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:18.862613 30233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:18.862656 30233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:18.862684 30233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:18.868898 30233 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:46725
I20260809 00:33:18.868938 30361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:46725 every 8 connection(s)
I20260809 00:33:18.869289 30233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:33:18.875357 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 30233
I20260809 00:33:18.875494 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:33:18.875818 21671 external_mini_cluster.cc:1428] Setting key 5d3a85988bf0a524c9bac319c4e7f5e7
Aug 09 00:33:18 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:18.879657 29961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:18.870067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:50779 (local address 127.21.41.254:46763)
0809 00:33:18.870127 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:18.870129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:18.870302 (+ 173us) server_negotiation.cc:407] Connection header received
0809 00:33:18.870892 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:18.870893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:18.870905 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:18.870976 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:18.872088 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.872679 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.873965 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.874159 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.874222 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:18.876954 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:18.876955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:18.876955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:18.876962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:18.878176 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:18.878649 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:18.878650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:18.878650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:18.878715 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:18.878992 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:18.878993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:18.878994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:18.879115 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:18.879186 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:18.879481 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 00:33:18.879567 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:33:18.879849 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:18.870013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:50779)
0809 00:33:18.870181 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:18.870225 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:18.870763 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:18.871108 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:18.871112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:18.871794 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:18.871985 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:18.871990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.872792 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.872793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:18.873841 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:18.873844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:18.874240 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:18.874240 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:18.874398 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:18.874967 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:18.874975 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:18.876867 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:18.878290 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:18.878296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:18.878298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:18.878568 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:18.878804 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:18.878805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:18.878805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:18.878919 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:18.879189 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:18.879196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:18.879341 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 00:33:18.879444 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 00:33:18.879607 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260809 00:33:18.880342 30362 heartbeater.cc:347] Connected to a master server at 127.21.41.254:46763
I20260809 00:33:18.880424 30362 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:18.880568 30362 heartbeater.cc:511] Master 127.21.41.254:46763 requested a full tablet report, sending...
I20260809 00:33:18.880913 29903 ts_manager.cc:194] Registered new tserver with Master: 38a949af19344a8da2684e7329875660 (127.21.41.195:46725)
I20260809 00:33:18.881332 29903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:50779
I20260809 00:33:18.887892 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab.
Entry for principal oryx/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab.
WARNING: no policy specified for oryx/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab.
Entry for principal oryx/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab.
WARNING: no policy specified for oryx/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab.
Entry for principal oryx/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab.
I20260809 00:33:18.965373 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 29965
I20260809 00:33:18.970183 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 30099
I20260809 00:33:18.974823 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 30233
I20260809 00:33:18.978984 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 29871
I20260809 00:33:18.984481 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:46763
--webserver_interface=127.21.41.254
--webserver_port=46615
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:46763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:19.068277 30371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:19.068528 30371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:19.068567 30371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:19.069752 30371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:19.069823 30371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:19.069847 30371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:19.069868 30371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:19.069898 30371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:19.071393 30371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:46763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:46763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=46615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:19.071683 30371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:19.071923 30371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:19.076826 30377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:19.076835 30379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:19.076838 30376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:19.077308 30371 server_base.cc:1034] running on GCE node
I20260809 00:33:19.077530 30371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:19.077773 30371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:19.078926 30371 hybrid_clock.cc:648] HybridClock initialized: now 1786235599078911 us; error 28 us; skew 500 ppm
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:19.081390 30371 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:33:19.081835 30371 webserver.cc:460] Webserver started at http://127.21.41.254:46615/ using document root <none> and password file <none>
I20260809 00:33:19.082062 30371 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:19.082144 30371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:19.083541 30371 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:19.084312 30386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:19.084496 30371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:19.084597 30371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "46fb58b4f9a04be9bbef5f548769101e"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "17c8cc16901dd05d5c032f5953b8bdcd"
server_key_iv: "4bd4d2eb8af13576fc2750f5e91f4b24"
server_key_version: "encryptionkey@0"
I20260809 00:33:19.084954 30371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:19.095168 30371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:19.097394 30371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:19.097572 30371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:19.101868 30371 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:46763
I20260809 00:33:19.101922 30438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:46763 every 8 connection(s)
I20260809 00:33:19.102322 30371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:19.104074 30439 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:33:19.104800 30439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e: Bootstrap starting.
I20260809 00:33:19.109664 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 30371
I20260809 00:33:19.110059 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:35855
--local_ip_for_outbound_sockets=127.21.41.193
--tserver_master_addrs=127.21.41.254:46763
--webserver_port=43843
--webserver_interface=127.21.41.193
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.41.193
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:33:19.114305 30439 log.cc:824] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:19.116228 30439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:33:19.116555 30439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e: Bootstrap complete.
I20260809 00:33:19.119131 30439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } }
I20260809 00:33:19.119370 30439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46fb58b4f9a04be9bbef5f548769101e, State: Initialized, Role: FOLLOWER
I20260809 00:33:19.119551 30439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } }
I20260809 00:33:19.119628 30439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:19.119658 30439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:19.119732 30439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:33:19.120800 30439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } }
I20260809 00:33:19.120893 30439 leader_election.cc:302] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46fb58b4f9a04be9bbef5f548769101e; no voters:
I20260809 00:33:19.121133 30439 leader_election.cc:288] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:33:19.121168 30443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:33:19.121315 30443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [term 2 LEADER]: Becoming Leader. State: Replica: 46fb58b4f9a04be9bbef5f548769101e, State: Running, Role: LEADER
I20260809 00:33:19.121431 30439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:19.121407 30443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } }
I20260809 00:33:19.121950 30445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46fb58b4f9a04be9bbef5f548769101e. Latest consensus state: current_term: 2 leader_uuid: "46fb58b4f9a04be9bbef5f548769101e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } } }
I20260809 00:33:19.122031 30445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:19.122244 30444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "46fb58b4f9a04be9bbef5f548769101e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fb58b4f9a04be9bbef5f548769101e" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 46763 } } }
I20260809 00:33:19.122372 30444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:19.122520 30451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:19.122996 30451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:19.123189 30451 catalog_manager.cc:1259] Loaded cluster ID: 38642032d3c8477a972098a3679d3f49
I20260809 00:33:19.123425 30451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:33:19.124717 30451 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:19.124897 30451 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 46fb58b4f9a04be9bbef5f548769101e: Loaded TSK: 0
I20260809 00:33:19.125144 30451 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:33:19.197643 30441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:19.197886 30441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:19.197930 30441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:19.199291 30441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:19.199360 30441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:19.199406 30441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:33:19.200966 30441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.193.keytab
--principal=oryx/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:35855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=43843
--tserver_master_addrs=127.21.41.254:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:19.201244 30441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:19.201489 30441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:19.206741 30465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:19.206769 30441 server_base.cc:1034] running on GCE node
W20260809 00:33:19.206746 30466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:19.206799 30468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:19.207209 30441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:19.207517 30441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:19.208731 30441 hybrid_clock.cc:648] HybridClock initialized: now 1786235599208721 us; error 32 us; skew 500 ppm
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235599, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:19.216822 30441 init.cc:377] Logged in from keytab as oryx/127.21.41.193@KRBTEST.COM (short username oryx)
I20260809 00:33:19.217501 30441 webserver.cc:460] Webserver started at http://127.21.41.193:43843/ using document root <none> and password file <none>
I20260809 00:33:19.217805 30441 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:19.217901 30441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:19.219451 30441 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:19.220148 30475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:19.220371 30441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:19.220486 30441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "e5a4fab2a16948119c4f71fc896b4f8b"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "2ffe06786fa3f949c4277e41f732f377"
server_key_iv: "50a12105d73534f8e5107e506d40174c"
server_key_version: "encryptionkey@0"
I20260809 00:33:19.220885 30441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:19.235416 30441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:19.237859 30441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:19.238029 30441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:19.238297 30441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:19.238631 30441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:19.238687 30441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:19.238729 30441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:19.238759 30441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:19.246301 30441 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:35855
I20260809 00:33:19.246407 30588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:35855 every 8 connection(s)
I20260809 00:33:19.246737 30441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:19.256067 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 30441
I20260809 00:33:19.256397 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:46071
--local_ip_for_outbound_sockets=127.21.41.194
--tserver_master_addrs=127.21.41.254:46763
--webserver_port=38229
--webserver_interface=127.21.41.194
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.41.194
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:19.257725 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:19.247493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:55699)
0809 00:33:19.247710 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:19.247763 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:19.249076 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:19.249915 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:19.249918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:19.250171 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:19.250426 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.250432 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.252227 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.252227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.253036 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.253038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.253943 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.253945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.254096 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.254672 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:19.254683 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:19.257487 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":115,"threads_started":1}
W20260809 00:33:19.258154 30592 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:19.247632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:55699 (local address 127.21.41.254:46763)
0809 00:33:19.249310 (+ 1678us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:19.249320 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:19.249360 (+ 40us) server_negotiation.cc:407] Connection header received
0809 00:33:19.249507 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:19.249514 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:19.249576 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:19.249863 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:19.250550 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.252120 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.253132 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.253821 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.254425 (+ 604us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.257683 (+ 3258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:55699: BlockingRecv error: recv got EOF from 127.21.41.193:55699 (error 108)
Metrics: {"server-negotiator.queue_time_us":1620,"thread_start_us":80,"threads_started":1}
W20260809 00:33:19.258188 30589 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:46763 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.41.254:46763: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:19.264570 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:19.258439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:34193)
0809 00:33:19.258485 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:19.258495 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:19.258530 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:19.258883 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:19.258884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:19.259047 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:19.259104 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.259106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.260055 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.260056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.261009 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.261011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.261628 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.261630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.261728 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.262276 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:19.262282 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:19.264492 (+ 2210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:33:19.264873 30594 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:19.258515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:34193 (local address 127.21.41.254:46763)
0809 00:33:19.258726 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:19.258728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:19.258737 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:19.258760 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:19.258760 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:19.258781 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:19.258851 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:19.259202 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.259950 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.261126 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.261519 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.261595 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.264609 (+ 3014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:34193: BlockingRecv error: recv got EOF from 127.21.41.193:34193 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":128,"threads_started":1}
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:19.270574 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:19.264824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:51705)
0809 00:33:19.265044 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:19.265055 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:19.265094 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:19.265289 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:19.265290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:19.265440 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:19.265475 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.265476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.266470 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.266471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.267370 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.267372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.267876 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.267877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.267935 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.268402 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:19.268407 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:19.270501 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":87,"threads_started":1}
W20260809 00:33:19.270673 30594 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:19.265003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:51705 (local address 127.21.41.254:46763)
0809 00:33:19.265111 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:19.265113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:19.265123 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:19.265147 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:19.265148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:19.265159 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:19.265259 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:19.265557 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.266389 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.267460 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.267791 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.267863 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.270613 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:51705: BlockingRecv error: recv got EOF from 127.21.41.193:51705 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260809 00:33:19.271028 30589 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:33:19.340835 30593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:19.341037 30593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:19.341060 30593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:19.342284 30593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:19.342329 30593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:19.342410 30593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:33:19.343856 30593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.194.keytab
--principal=oryx/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:46071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=38229
--tserver_master_addrs=127.21.41.254:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:19.344087 30593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:19.344340 30593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:19.349431 30600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:19.349473 30603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:19.349444 30601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:19.349881 30593 server_base.cc:1034] running on GCE node
I20260809 00:33:19.350096 30593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:19.350375 30593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:19.351585 30593 hybrid_clock.cc:648] HybridClock initialized: now 1786235599351569 us; error 39 us; skew 500 ppm
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235599, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:19.354252 30593 init.cc:377] Logged in from keytab as oryx/127.21.41.194@KRBTEST.COM (short username oryx)
I20260809 00:33:19.354748 30593 webserver.cc:460] Webserver started at http://127.21.41.194:38229/ using document root <none> and password file <none>
I20260809 00:33:19.355024 30593 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:19.355110 30593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:19.356698 30593 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:33:19.357264 30610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:19.357434 30593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:19.357530 30593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "ed19688e5875496f97868cf69e34bb9f"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "67855b3250b8b0214d9e1424f7b3fdf1"
server_key_iv: "031d43c4c21b987780787161702e0284"
server_key_version: "encryptionkey@0"
I20260809 00:33:19.357918 30593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:19.377310 30593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:19.379333 30593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:19.379501 30593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:19.379763 30593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:19.380079 30593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:19.380133 30593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:19.380178 30593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:19.380211 30593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:19.386296 30593 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:46071
I20260809 00:33:19.386400 30723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:46071 every 8 connection(s)
I20260809 00:33:19.386744 30593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:33:19.392457 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 30593
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:19.392772 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:46725
--local_ip_for_outbound_sockets=127.21.41.195
--tserver_master_addrs=127.21.41.254:46763
--webserver_port=41925
--webserver_interface=127.21.41.195
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.41.195
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 00:33:19.394497 30594 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:19.387434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:36767 (local address 127.21.41.254:46763)
0809 00:33:19.387477 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:19.387479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:19.387728 (+ 249us) server_negotiation.cc:407] Connection header received
0809 00:33:19.388226 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:19.388227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:19.388242 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:19.388310 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:19.389266 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.389753 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.390854 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.391035 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.391089 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.394413 (+ 3324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:36767: BlockingRecv error: recv got EOF from 127.21.41.194:36767 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:33:19.394531 30726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:19.387468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:36767)
0809 00:33:19.387621 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:19.387657 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:19.388142 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:19.388366 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:19.388369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:19.388982 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:19.389180 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.389185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.389856 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.389857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.390779 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.390782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.391120 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.391121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.391229 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.391712 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:19.391720 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:19.394259 (+ 2539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
W20260809 00:33:19.394797 30724 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:46763 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.41.254:46763: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:19.400218 30726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:19.395075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:37959)
0809 00:33:19.395106 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:19.395118 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:19.395158 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:19.395326 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:19.395327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:19.395437 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:19.395484 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.395485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.396391 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.396392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.397030 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.397032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.397549 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.397550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.397663 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.398122 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:19.398126 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:19.400119 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 00:33:19.400249 30594 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:19.395132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:37959 (local address 127.21.41.254:46763)
0809 00:33:19.395189 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:19.395192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:19.395201 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:19.395224 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:19.395225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:19.395238 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:19.395318 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:19.395556 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.396307 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.397122 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.397465 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.397538 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.400204 (+ 2666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:37959: BlockingRecv error: recv got EOF from 127.21.41.194:37959 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:19.405318 30726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:19.400622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:38013)
0809 00:33:19.400738 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:19.400750 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:19.400807 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:19.400949 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:19.400950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:19.401100 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:19.401140 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.401141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.401757 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.401759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.402656 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.402659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.403027 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.403028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.403087 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.403541 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:19.403545 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:19.405239 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98}
W20260809 00:33:19.405369 30594 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:19.400691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:38013 (local address 127.21.41.254:46763)
0809 00:33:19.400744 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:19.400745 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:19.400800 (+ 55us) server_negotiation.cc:407] Connection header received
0809 00:33:19.400892 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:19.400893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:19.400900 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:19.400944 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:19.401212 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.401677 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.402747 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.402923 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.403026 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.405308 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:38013: BlockingRecv error: recv got EOF from 127.21.41.194:38013 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 00:33:19.405449 30724 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:33:19.477021 30727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:19.477258 30727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:19.477296 30727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:19.478621 30727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:19.478680 30727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:19.478724 30727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:19.480180 30727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/krb5kdc/oryx_127.21.41.195.keytab
--principal=oryx/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:46725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=41925
--tserver_master_addrs=127.21.41.254:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:19.480458 30727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:19.480687 30727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:19.485707 30733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:19.485724 30732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:19.485778 30735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:19.485782 30727 server_base.cc:1034] running on GCE node
I20260809 00:33:19.486120 30727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:19.486315 30727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:19.487512 30727 hybrid_clock.cc:648] HybridClock initialized: now 1786235599487494 us; error 34 us; skew 500 ppm
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235599, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:19.490046 30727 init.cc:377] Logged in from keytab as oryx/127.21.41.195@KRBTEST.COM (short username oryx)
I20260809 00:33:19.490550 30727 webserver.cc:460] Webserver started at http://127.21.41.195:41925/ using document root <none> and password file <none>
I20260809 00:33:19.490844 30727 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:19.490927 30727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:19.492233 30727 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:19.492810 30742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:19.492966 30727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:19.493064 30727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "38a949af19344a8da2684e7329875660"
format_stamp: "Formatted at 2026-08-09 00:33:18 on dist-test-slave-zpfg"
server_key: "7710afb2a1da8f0ee390e933eecddfcd"
server_key_iv: "45edd7c17df8d42f17010c911a823a5b"
server_key_version: "encryptionkey@0"
I20260809 00:33:19.493438 30727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:19.516744 30727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:19.518720 30727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:19.518888 30727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:19.519134 30727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:19.519449 30727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:19.519503 30727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:19.519549 30727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:19.519573 30727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:19.525899 30727 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:46725
I20260809 00:33:19.525943 30855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:46725 every 8 connection(s)
I20260809 00:33:19.526294 30727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:33:19.528797 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 30727
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 00:33:19.534204 30594 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:19.527122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:51987 (local address 127.21.41.254:46763)
0809 00:33:19.527199 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:19.527202 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:19.527347 (+ 145us) server_negotiation.cc:407] Connection header received
0809 00:33:19.527811 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:19.527813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:19.527822 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:19.527876 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:19.528799 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.529448 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.530614 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.530826 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.530883 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.534143 (+ 3260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:51987: BlockingRecv error: recv got EOF from 127.21.41.195:51987 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 00:33:19.534284 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:19.527064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:51987)
0809 00:33:19.527222 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:19.527259 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:19.527717 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:19.527930 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:19.527933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:19.528497 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:19.528693 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.528698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.529542 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.529543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.530537 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.530539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.530906 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.530907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.531062 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.531594 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:19.531602 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:19.534074 (+ 2472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":96,"threads_started":1}
W20260809 00:33:19.534540 30856 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:46763 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.41.254:46763: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
I20260809 00:33:19.536939 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:19.529346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51388 (local address 127.21.41.254:46763)
0809 00:33:19.529412 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:19.529414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:19.529461 (+ 47us) server_negotiation.cc:407] Connection header received
0809 00:33:19.529542 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:19.529543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:19.529556 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:19.529635 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:19.530283 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.530919 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.531889 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.532202 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.532269 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.533576 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:19.533580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:19.533580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:19.533658 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:19.535460 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:19.535851 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:19.535853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:19.535854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:19.535937 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:19.536155 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:19.536155 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:19.536156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:19.536430 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:19.536564 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:19.536732 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 00:33:19.536841 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:19.539585 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:19.534806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:50927)
0809 00:33:19.534898 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:19.534909 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:19.534955 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:19.535082 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:19.535082 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:19.535225 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:19.535268 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.535270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.536094 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.536094 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.536945 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.536947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.537421 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.537422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.537491 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.537903 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:19.537908 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:19.539541 (+ 1633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":80}
W20260809 00:33:19.539769 30594 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:19.534843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:50927 (local address 127.21.41.254:46763)
0809 00:33:19.534902 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:19.534904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:19.534941 (+ 37us) server_negotiation.cc:407] Connection header received
0809 00:33:19.535010 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:19.535017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:19.535026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:19.535081 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:19.535344 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.536022 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.537016 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.537340 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.537422 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.539709 (+ 2287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:50927: BlockingRecv error: recv got EOF from 127.21.41.195:50927 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:19 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:19.543767 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:19.539861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:33527)
0809 00:33:19.539894 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:19.539905 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:19.539941 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:19.540107 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:19.540107 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:19.540228 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:19.540252 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.540253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.540838 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.540839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.541343 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:19.541345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.541739 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.541740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:19.541792 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.542134 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:19.542138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:19.543725 (+ 1587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 00:33:19.543862 30594 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:19.539943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:33527 (local address 127.21.41.254:46763)
0809 00:33:19.539981 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:19.539982 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:19.539990 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:33:19.540006 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:19.540006 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:19.540013 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:19.540102 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:19.540323 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.540754 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.541427 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:19.541630 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:19.541728 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:19.543799 (+ 2071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:33527: BlockingRecv error: recv got EOF from 127.21.41.195:33527 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 00:33:19.543942 30856 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 00:33:20 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:20 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:20.276347 30860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:20.271464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:58767)
0809 00:33:20.271612 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:20.271631 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:20.271684 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:20.271835 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:20.271836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:20.272004 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:20.272055 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:20.272056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.272751 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.272752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:20.273345 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:20.273348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.273697 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.273698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:20.273751 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:20.274248 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:20.274253 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:20.276222 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
W20260809 00:33:20.276388 30861 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:20.271524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:58767 (local address 127.21.41.254:46763)
0809 00:33:20.271676 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:20.271678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:20.271687 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:20.271730 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:20.271730 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:20.271738 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:20.271813 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:20.272128 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.272643 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.273428 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.273620 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.273673 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:20.276311 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:58767: BlockingRecv error: recv got EOF from 127.21.41.193:58767 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
Aug 09 00:33:20 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:20 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:20.410956 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:20.405959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:59533)
0809 00:33:20.406109 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:20.406123 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:20.406198 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:20.406399 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:20.406400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:20.406584 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:20.406632 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:20.406634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.407309 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.407310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:20.407849 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:20.407850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.408234 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.408235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:20.408304 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:20.408779 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:20.408784 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:20.410887 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
W20260809 00:33:20.411022 30861 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:20.406080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:59533 (local address 127.21.41.254:46763)
0809 00:33:20.406149 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:20.406151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:20.406178 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:33:20.406276 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:20.406277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:20.406291 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:20.406388 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:20.406722 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.407221 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.407932 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.408128 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.408233 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:20.410976 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:59533: BlockingRecv error: recv got EOF from 127.21.41.194:59533 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 00:33:20 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:20 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:20.549247 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:20.544381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:36627)
0809 00:33:20.544571 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:20.544583 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:20.544732 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:20.544889 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:20.544890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:20.545073 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:20.545124 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:20.545125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.545796 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.545798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:20.546370 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:20.546372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.546755 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.546756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:20.546816 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:20.547260 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:20.547265 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:20.549138 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":99,"threads_started":1}
W20260809 00:33:20.549283 30861 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:20.544470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:36627 (local address 127.21.41.254:46763)
0809 00:33:20.544564 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:20.544566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:20.544713 (+ 147us) server_negotiation.cc:407] Connection header received
0809 00:33:20.544820 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:20.544821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:20.544836 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:20.544885 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:20.545210 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.545711 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.546460 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:20.546664 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:20.546753 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:20.549217 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:36627: BlockingRecv error: recv got EOF from 127.21.41.195:36627 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 00:33:21 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:21 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:21.281524 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:21.276895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:38299)
0809 00:33:21.277094 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:21.277108 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:21.277178 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:21.277322 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:21.277323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:21.277485 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:21.277535 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:21.277536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.278184 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.278185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:21.278849 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:21.278851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.279168 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.279169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:21.279221 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:21.279688 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:21.279693 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:21.281467 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":119,"threads_started":1}
W20260809 00:33:21.281674 30865 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:21.277019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:38299 (local address 127.21.41.254:46763)
0809 00:33:21.277126 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:21.277128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:21.277160 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:33:21.277236 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:21.277236 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:21.277247 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:21.277304 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:21.277597 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.278113 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.278920 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.279106 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.279157 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:21.281583 (+ 2426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:38299: BlockingRecv error: recv got EOF from 127.21.41.193:38299 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":53,"threads_started":1}
Aug 09 00:33:21 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:21 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:21.416597 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:21.411454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:39703)
0809 00:33:21.411692 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:21.411704 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:21.411770 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:21.411936 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:21.411937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:21.412091 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:21.412140 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:21.412142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.412884 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.412884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:21.413398 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:21.413399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.413832 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.413833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:21.413874 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:21.414406 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:21.414412 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:21.416539 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":190,"threads_started":1}
W20260809 00:33:21.416755 30865 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:21.411549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:39703 (local address 127.21.41.254:46763)
0809 00:33:21.411696 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:21.411698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:21.411775 (+ 77us) server_negotiation.cc:407] Connection header received
0809 00:33:21.411859 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:21.411860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:21.411872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:21.411920 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:21.412213 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.412812 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.413475 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.413747 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.413810 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:21.416660 (+ 2850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:39703: BlockingRecv error: recv got EOF from 127.21.41.194:39703 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 00:33:21 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:21 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:21.555248 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:21.549717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:51255)
0809 00:33:21.549885 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:21.549900 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:21.549978 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:21.550168 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:21.550170 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:21.550392 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:21.550478 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:21.550480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.551225 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.551226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:21.551856 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:21.551858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.552273 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.552274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:21.552341 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:21.552804 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:21.552809 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:21.555173 (+ 2364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
W20260809 00:33:21.555323 30865 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:21.549860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:51255 (local address 127.21.41.254:46763)
0809 00:33:21.549905 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:21.549908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:21.549956 (+ 48us) server_negotiation.cc:407] Connection header received
0809 00:33:21.550038 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:21.550039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:21.550054 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:21.550162 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:21.550559 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.551129 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.551955 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:21.552175 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:21.552271 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:21.555257 (+ 2986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:51255: BlockingRecv error: recv got EOF from 127.21.41.195:51255 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 00:33:22 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:22 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:22.287456 30868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:22.282087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:47163)
0809 00:33:22.282275 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:22.282286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:22.282401 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:22.282574 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:22.282575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:22.282778 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:22.282829 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:22.282831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.283551 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.283552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:22.284197 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:22.284200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.284611 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.284612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:22.284670 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:22.285236 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:22.285241 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:22.287346 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
W20260809 00:33:22.287628 30869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:22.282201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:47163 (local address 127.21.41.254:46763)
0809 00:33:22.282405 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:22.282407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:22.282451 (+ 44us) server_negotiation.cc:407] Connection header received
0809 00:33:22.282475 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:22.282476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:22.282487 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:22.282543 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:22.282904 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.283469 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.284287 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.284501 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.284561 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:22.287544 (+ 2983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:47163: BlockingRecv error: recv got EOF from 127.21.41.193:47163 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
Aug 09 00:33:22 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:22 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:22.422796 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:22.417156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:39267)
0809 00:33:22.417326 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:22.417337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:22.417400 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:22.417533 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:22.417534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:22.417720 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:22.417773 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:22.417775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.418775 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.418777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:22.419370 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:22.419372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.419801 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.419802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:22.419873 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:22.420438 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:22.420443 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:22.422728 (+ 2285us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
W20260809 00:33:22.422923 30869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:22.417242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:39267 (local address 127.21.41.254:46763)
0809 00:33:22.417324 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:22.417326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:22.417398 (+ 72us) server_negotiation.cc:407] Connection header received
0809 00:33:22.417460 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:22.417460 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:22.417472 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:22.417527 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:22.417872 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.418666 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.419468 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.419697 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.419799 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:22.422848 (+ 3049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:39267: BlockingRecv error: recv got EOF from 127.21.41.194:39267 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 00:33:22 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:22 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:22.561167 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:22.555878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:34371)
0809 00:33:22.556033 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:22.556068 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:22.556132 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:22.556269 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:22.556269 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:22.556455 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:22.556503 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:22.556504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.557284 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.557285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:22.557819 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:22.557821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.558234 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.558235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:22.558301 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:22.558835 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:22.558840 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:22.561068 (+ 2228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
W20260809 00:33:22.561419 30869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:22.556020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:34371 (local address 127.21.41.254:46763)
0809 00:33:22.556129 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:22.556132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:22.556143 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:33:22.556195 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:22.556196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:22.556212 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:22.556264 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:22.556671 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.557194 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.557927 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:22.558136 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:22.558231 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:22.561202 (+ 2971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:34371: BlockingRecv error: recv got EOF from 127.21.41.195:34371 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 00:33:23 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:23 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:23.292466 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:23.287934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:57929)
0809 00:33:23.288124 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:23.288135 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:23.288197 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:23.288326 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:23.288326 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:23.288479 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:23.288529 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:23.288530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.289165 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.289166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:23.289686 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:23.289687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.290023 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.290024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:23.290063 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:23.290582 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:23.290586 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:23.292407 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":104,"threads_started":1}
W20260809 00:33:23.292605 30873 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:23.288012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:57929 (local address 127.21.41.254:46763)
0809 00:33:23.288155 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:23.288157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:23.288200 (+ 43us) server_negotiation.cc:407] Connection header received
0809 00:33:23.288240 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:23.288241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:23.288251 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:23.288312 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:23.288591 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.289079 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.289756 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.289957 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.290012 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:23.292493 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:57929: BlockingRecv error: recv got EOF from 127.21.41.193:57929 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
Aug 09 00:33:23 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:23 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:23.428960 30874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:23.423440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:57757)
0809 00:33:23.423591 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:23.423604 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:23.423722 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:23.423946 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:23.423947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:23.424236 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:23.424302 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:23.424305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.425199 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.425200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:23.425801 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:23.425803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.426222 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.426223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:23.426288 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:23.426820 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:23.426825 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:23.428850 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
W20260809 00:33:23.429024 30873 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:23.423524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:57757 (local address 127.21.41.254:46763)
0809 00:33:23.423585 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:23.423588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:23.423716 (+ 128us) server_negotiation.cc:407] Connection header received
0809 00:33:23.423804 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:23.423805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:23.423818 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:23.423941 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:23.424373 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.425063 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.425903 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.426124 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.426219 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:23.428951 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:57757: BlockingRecv error: recv got EOF from 127.21.41.194:57757 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 09 00:33:23 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:23 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:23.567019 30875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:23.561812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:36469)
0809 00:33:23.561984 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:23.561998 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:23.562061 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:23.562304 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:23.562306 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:23.562507 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:23.562559 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:23.562561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.563255 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.563256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:23.563785 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:23.563787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.564124 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.564125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:23.564168 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:23.564820 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:23.564825 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:23.566951 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":110,"threads_started":1}
W20260809 00:33:23.567157 30873 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:23.561988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:36469 (local address 127.21.41.254:46763)
0809 00:33:23.562051 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:23.562053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:23.562063 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:23.562184 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:23.562185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:23.562196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:23.562253 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:23.562647 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.563139 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.563869 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:23.564052 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:23.564107 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:23.567047 (+ 2940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:36469: BlockingRecv error: recv got EOF from 127.21.41.195:36469 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 00:33:24 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:24 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:24.303090 30876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:24.292947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:40683)
0809 00:33:24.293139 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:24.293151 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:24.293230 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:24.293409 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:24.293410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:24.293581 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:24.293634 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:24.293636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.294268 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.294269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:24.294923 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:24.294925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.295246 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.295247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:24.295289 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:24.300964 (+ 5675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:24.300970 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:24.303016 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":124,"threads_started":1}
W20260809 00:33:24.303236 30877 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:24.293021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:40683 (local address 127.21.41.254:46763)
0809 00:33:24.293156 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:24.293158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:24.293233 (+ 75us) server_negotiation.cc:407] Connection header received
0809 00:33:24.293282 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:24.293283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:24.293293 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:24.293355 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:24.293704 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.294194 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.294993 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.295180 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.295231 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:24.303121 (+ 7890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:40683: BlockingRecv error: recv got EOF from 127.21.41.193:40683 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
Aug 09 00:33:24 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:24 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:24.434129 30878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:24.429506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:54383)
0809 00:33:24.429659 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:24.429673 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:24.429766 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:24.429899 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:24.429899 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:24.430061 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:24.430112 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:24.430114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.430853 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.430854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:24.431398 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:24.431400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.431769 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.431770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:24.431833 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:24.432309 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:24.432313 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:24.434067 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
W20260809 00:33:24.434237 30877 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:24.429659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:54383 (local address 127.21.41.254:46763)
0809 00:33:24.429723 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:24.429725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:24.429750 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:33:24.429820 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:24.429821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:24.429830 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:24.429880 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:24.430175 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.430720 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.431492 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.431692 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.431770 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:24.434165 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:54383: BlockingRecv error: recv got EOF from 127.21.41.194:54383 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 00:33:24 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:24 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:24.572371 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:24.567571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:33425)
0809 00:33:24.567710 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:24.567728 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:24.567780 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:24.567917 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:24.567918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:24.568078 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:24.568127 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:24.568129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.568843 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.568844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:24.569375 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:24.569376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.569727 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.569727 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:24.569765 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:24.570293 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:24.570297 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:24.572293 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
W20260809 00:33:24.572539 30877 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:24.567699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:33425 (local address 127.21.41.254:46763)
0809 00:33:24.567762 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:24.567764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:24.567781 (+ 17us) server_negotiation.cc:407] Connection header received
0809 00:33:24.567828 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:24.567829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:24.567840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:24.567899 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:24.568193 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.568741 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.569455 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:24.569658 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:24.569718 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:24.572461 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:33425: BlockingRecv error: recv got EOF from 127.21.41.195:33425 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 00:33:25 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:25 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:25.308725 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:25.303601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:56049)
0809 00:33:25.303751 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:25.303761 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:25.303835 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:25.304026 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:25.304026 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:25.304185 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:25.304235 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:25.304236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.305084 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.305085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:25.305700 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:25.305702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.306066 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.306067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:25.306108 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:25.306593 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:25.306598 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:25.308617 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
W20260809 00:33:25.308792 30881 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:25.303737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:56049 (local address 127.21.41.254:46763)
0809 00:33:25.303880 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:25.303882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:25.303892 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:25.303917 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:25.303917 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:25.303928 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:25.304008 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:25.304310 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.304976 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.305778 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.305991 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.306050 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:25.308712 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:56049: BlockingRecv error: recv got EOF from 127.21.41.193:56049 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
Aug 09 00:33:25 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:25 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:25.439743 30882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:25.434649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:42289)
0809 00:33:25.434857 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:25.434869 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:25.434941 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:25.435117 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:25.435119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:25.435295 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:25.435347 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:25.435349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.436221 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.436222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:25.436872 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:25.436874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.437212 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.437213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:25.437254 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:25.437802 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:25.437807 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:25.439678 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":129,"threads_started":1}
W20260809 00:33:25.439867 30881 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:25.434756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:42289 (local address 127.21.41.254:46763)
0809 00:33:25.434827 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:25.434829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:25.434924 (+ 95us) server_negotiation.cc:407] Connection header received
0809 00:33:25.434991 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:25.434992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:25.435002 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:25.435059 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:25.435417 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.436114 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.436956 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.437138 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.437192 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:25.439766 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:42289: BlockingRecv error: recv got EOF from 127.21.41.194:42289 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 00:33:25 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:25 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:25.577477 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:25.572853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:49853)
0809 00:33:25.572982 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:25.572996 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:25.573069 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:25.573216 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:25.573218 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:25.573410 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:25.573458 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:25.573460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.574125 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.574127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:25.574705 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:25.574707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.575071 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.575072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:25.575126 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:25.575624 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:25.575630 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:25.577377 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
W20260809 00:33:25.577523 30881 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:25.572965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:49853 (local address 127.21.41.254:46763)
0809 00:33:25.573022 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:25.573024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:25.573049 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:33:25.573126 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:25.573127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:25.573147 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:25.573211 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:25.573547 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.574042 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.574794 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:25.574985 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:25.575071 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:25.577458 (+ 2387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:49853: BlockingRecv error: recv got EOF from 127.21.41.195:49853 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 09 00:33:26 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:26 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:26.314280 30884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:26.309306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:36969)
0809 00:33:26.309504 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:26.309518 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:26.309590 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:26.309845 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:26.309847 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:26.310014 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:26.310064 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:26.310066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.310889 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.310890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:26.311456 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:26.311458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.311892 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.311893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:26.311932 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:26.312465 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:26.312470 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:26.314219 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
W20260809 00:33:26.314457 30885 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:26.309422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:36969 (local address 127.21.41.254:46763)
0809 00:33:26.309569 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:26.309571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:26.309582 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:33:26.309706 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:26.309707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:26.309719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:26.309801 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:26.310144 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.310811 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.311552 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.311818 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.311881 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:26.314344 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:36969: BlockingRecv error: recv got EOF from 127.21.41.193:36969 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
Aug 09 00:33:26 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:26 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:26.444671 30886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:26.440248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:50755)
0809 00:33:26.440389 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:26.440400 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:26.440470 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:26.440604 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:26.440605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:26.440761 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:26.440820 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:26.440821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.441453 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.441454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:26.441988 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:26.441990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.442333 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.442334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:26.442403 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:26.442879 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:26.442884 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:26.444611 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
W20260809 00:33:26.444813 30885 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:26.440341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:50755 (local address 127.21.41.254:46763)
0809 00:33:26.440390 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:26.440392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:26.440455 (+ 63us) server_negotiation.cc:407] Connection header received
0809 00:33:26.440532 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:26.440533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:26.440544 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:26.440585 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:26.440885 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.441383 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.442065 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.442265 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.442319 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:26.444723 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:50755: BlockingRecv error: recv got EOF from 127.21.41.194:50755 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 00:33:26 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:26 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:26.583266 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:26.578071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:40651)
0809 00:33:26.578292 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:26.578304 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:26.578427 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:26.578664 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:26.578666 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:26.578877 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:26.578970 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:26.578972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.579756 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.579757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:26.580320 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:26.580322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.580698 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.580699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:26.580745 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:26.581211 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:26.581215 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:26.583187 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":118,"threads_started":1}
W20260809 00:33:26.583390 30885 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:26.578170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:40651 (local address 127.21.41.254:46763)
0809 00:33:26.578271 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:26.578286 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:26.578394 (+ 108us) server_negotiation.cc:407] Connection header received
0809 00:33:26.578569 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:26.578570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:26.578581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:26.578653 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:26.579070 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.579686 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.580423 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:26.580611 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:26.580689 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:26.583289 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:40651: BlockingRecv error: recv got EOF from 127.21.41.195:40651 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 00:33:27 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:27 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:27.319792 30888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:27.314808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:51957)
0809 00:33:27.314963 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:27.314974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:27.315027 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:27.315274 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:27.315276 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:27.315455 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:27.315506 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:27.315508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.316222 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.316223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:27.316813 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:27.316815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.317158 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.317159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:27.317200 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:27.317691 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:27.317696 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:27.319675 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
W20260809 00:33:27.319842 30889 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:27.314954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:51957 (local address 127.21.41.254:46763)
0809 00:33:27.315094 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:27.315096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:27.315103 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:33:27.315127 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:27.315128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:27.315138 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:27.315198 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:27.315589 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.316144 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.316895 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.317087 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.317146 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:27.319765 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:51957: BlockingRecv error: recv got EOF from 127.21.41.193:51957 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
Aug 09 00:33:27 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:27 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:27.450205 30890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:27.445305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:56145)
0809 00:33:27.445475 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:27.445496 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:27.445565 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:27.445772 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:27.445773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:27.445948 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:27.445999 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:27.446000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.446850 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.446851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:27.447376 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:27.447377 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.447773 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.447779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:27.447820 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:27.448271 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:27.448275 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:27.450130 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
W20260809 00:33:27.450317 30889 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:27.445364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:56145 (local address 127.21.41.254:46763)
0809 00:33:27.445444 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:27.445446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:27.445566 (+ 120us) server_negotiation.cc:407] Connection header received
0809 00:33:27.445621 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:27.445622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:27.445707 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:27.445757 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:27.446063 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.446777 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.447458 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.447702 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.447760 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:27.450237 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:56145: BlockingRecv error: recv got EOF from 127.21.41.194:56145 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 00:33:27 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:27 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:27.588562 30891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:27.583886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:58021)
0809 00:33:27.584046 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:27.584056 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:27.584134 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:27.584261 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:27.584261 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:27.584446 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:27.584493 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:27.584495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.585178 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.585179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:27.585699 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:27.585701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.586073 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.586074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:27.586131 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:27.586574 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:27.586578 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:27.588456 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
W20260809 00:33:27.588719 30889 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:27.583925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:58021 (local address 127.21.41.254:46763)
0809 00:33:27.583974 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:27.583976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:27.584119 (+ 143us) server_negotiation.cc:407] Connection header received
0809 00:33:27.584188 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:27.584189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:27.584204 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:27.584256 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:27.584574 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.585085 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.585784 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:27.585999 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:27.586076 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:27.588579 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:58021: BlockingRecv error: recv got EOF from 127.21.41.195:58021 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 00:33:28 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:28 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:28.325939 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:28.320620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:57283)
0809 00:33:28.320781 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:28.320794 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:28.320867 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:28.321093 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:28.321093 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:28.321398 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:28.321464 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:28.321466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.322209 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.322213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:28.322842 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:28.322844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.323284 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.323285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:28.323355 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:28.323911 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:28.323917 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:28.325874 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
W20260809 00:33:28.326046 30893 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:28.320760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:57283 (local address 127.21.41.254:46763)
0809 00:33:28.320914 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:28.320917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:28.320927 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:28.320954 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:28.320954 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:28.320968 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:28.321091 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:28.321528 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.322083 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.322924 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.323160 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.323212 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:28.325970 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:57283: BlockingRecv error: recv got EOF from 127.21.41.193:57283 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
Aug 09 00:33:28 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:28 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:28.456497 30894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:28.450918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:39899)
0809 00:33:28.451249 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:28.451266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:28.451358 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:28.451536 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:28.451537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:28.451795 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:28.451849 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:28.451851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.452611 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.452612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:28.453422 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:28.453426 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.453869 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.453870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:28.453917 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:28.454523 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:28.454528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:28.456435 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":207,"threads_started":1}
W20260809 00:33:28.456683 30893 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:28.451139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:39899 (local address 127.21.41.254:46763)
0809 00:33:28.451255 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:28.451257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:28.451334 (+ 77us) server_negotiation.cc:407] Connection header received
0809 00:33:28.451427 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:28.451428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:28.451442 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:28.451528 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:28.451952 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.452509 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.453513 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.453800 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.453857 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:28.456591 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:39899: BlockingRecv error: recv got EOF from 127.21.41.194:39899 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 00:33:28 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:28 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:28.594013 30895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:28.588996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:50923)
0809 00:33:28.589181 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:28.589195 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:28.589302 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:28.589440 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:28.589441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:28.589654 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:28.589702 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:28.589704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.590452 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.590453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:28.590999 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:28.591001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.591399 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.591400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:28.591464 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:28.592063 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:28.592068 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:28.593949 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
W20260809 00:33:28.594087 30893 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:28.589113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:50923 (local address 127.21.41.254:46763)
0809 00:33:28.589187 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:28.589189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:28.589282 (+ 93us) server_negotiation.cc:407] Connection header received
0809 00:33:28.589361 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:28.589362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:28.589376 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:28.589422 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:28.589789 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.590315 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.591093 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:28.591309 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:28.591397 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:28.594039 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:50923: BlockingRecv error: recv got EOF from 127.21.41.195:50923 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 00:33:29 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:29 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:29.335314 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:29.326449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:48071)
0809 00:33:29.326610 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:29.326621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:29.326682 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:29.326828 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:29.326829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:29.326991 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:29.327042 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:29.327043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.327728 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.327728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:29.328246 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:29.328248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.328591 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.328592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:29.328631 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:29.333155 (+ 4524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:29.333161 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:29.335214 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
W20260809 00:33:29.335371 30897 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:29.326521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:48071 (local address 127.21.41.254:46763)
0809 00:33:29.326633 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:29.326634 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:29.326686 (+ 52us) server_negotiation.cc:407] Connection header received
0809 00:33:29.326730 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:29.326731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:29.326742 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:29.326801 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:29.327113 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.327650 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.328324 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.328525 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.328580 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:29.335295 (+ 6715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:48071: BlockingRecv error: recv got EOF from 127.21.41.193:48071 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":50,"threads_started":1}
Aug 09 00:33:29 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:29 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:29.462234 30898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:29.457015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:60885)
0809 00:33:29.457207 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:29.457222 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:29.457319 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:29.457477 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:29.457477 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:29.457661 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:29.457709 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:29.457711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.458471 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.458472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:29.459013 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:29.459014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.459396 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.459397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:29.459462 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:29.459952 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:29.459956 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:29.462147 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":120,"threads_started":1}
W20260809 00:33:29.462400 30897 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:29.457152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:60885 (local address 127.21.41.254:46763)
0809 00:33:29.457230 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:29.457233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:29.457300 (+ 67us) server_negotiation.cc:407] Connection header received
0809 00:33:29.457413 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:29.457414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:29.457426 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:29.457472 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:29.457824 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.458396 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.459099 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.459319 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.459398 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:29.462240 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:60885: BlockingRecv error: recv got EOF from 127.21.41.194:60885 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 00:33:29 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:29 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:29.599427 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:29.594614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:50647)
0809 00:33:29.594853 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:29.594866 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:29.594949 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:29.595104 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:29.595105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:29.595283 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:29.595333 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:29.595335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.596131 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.596132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:29.596676 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:29.596677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.597086 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.597086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:29.597151 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:29.597586 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:29.597590 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:29.599322 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":162,"threads_started":1}
W20260809 00:33:29.599479 30897 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:29.594750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:50647 (local address 127.21.41.254:46763)
0809 00:33:29.594839 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:29.594842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:29.594928 (+ 86us) server_negotiation.cc:407] Connection header received
0809 00:33:29.595025 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:29.595026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:29.595040 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:29.595088 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:29.595412 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.596052 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.596771 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:29.597019 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:29.597085 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:29.599411 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:50647: BlockingRecv error: recv got EOF from 127.21.41.195:50647 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 00:33:30 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:30 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:30.340561 30900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:30.335852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:51853)
0809 00:33:30.336012 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:30.336023 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:30.336098 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:30.336231 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:30.336232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:30.336402 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:30.336454 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:30.336455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.337145 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.337146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:30.337686 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:30.337688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.338034 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.338035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:30.338080 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:30.338752 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:30.338757 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:30.340492 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
W20260809 00:33:30.340740 30901 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:30.335940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:51853 (local address 127.21.41.254:46763)
0809 00:33:30.336046 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:30.336048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:30.336076 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:33:30.336152 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:30.336152 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:30.336162 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:30.336209 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:30.336519 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.337065 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.337756 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.337965 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.338018 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:30.340643 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:51853: BlockingRecv error: recv got EOF from 127.21.41.193:51853 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":50,"threads_started":1}
Aug 09 00:33:30 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:30 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:30.468016 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:30.462779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:38839)
0809 00:33:30.462972 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:30.462985 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:30.463085 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:30.463268 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:30.463269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:30.463500 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:30.463578 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:30.463580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.464479 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.464480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:30.465033 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:30.465035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.465419 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.465419 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:30.465463 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:30.465967 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:30.465972 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:30.467946 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":107,"threads_started":1}
W20260809 00:33:30.468204 30901 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:30.462885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:38839 (local address 127.21.41.254:46763)
0809 00:33:30.462973 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:30.462975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:30.463068 (+ 93us) server_negotiation.cc:407] Connection header received
0809 00:33:30.463173 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:30.463174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:30.463187 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:30.463262 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:30.463702 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.464405 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.465152 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.465345 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.465409 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:30.468092 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:38839: BlockingRecv error: recv got EOF from 127.21.41.194:38839 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 00:33:30 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:30 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:30.605242 30903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:30.600056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:41139)
0809 00:33:30.600226 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:30.600240 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:30.600318 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:30.600506 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:30.600508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:30.600705 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:30.600748 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:30.600751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.601444 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.601445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:30.602018 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:30.602019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.602439 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.602440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:30.602507 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:30.603070 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:30.603075 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:30.605167 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
W20260809 00:33:30.605378 30901 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:30.600189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:41139 (local address 127.21.41.254:46763)
0809 00:33:30.600271 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:30.600273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:30.600287 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:33:30.600409 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:30.600410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:30.600426 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:30.600499 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:30.600834 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.601347 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.602128 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:30.602313 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:30.602438 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:30.605273 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:41139: BlockingRecv error: recv got EOF from 127.21.41.195:41139 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 00:33:31 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:31 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:31.345511 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:31.341057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:45621)
0809 00:33:31.341277 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:31.341287 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:31.341342 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:31.341459 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:31.341460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:31.341623 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:31.341722 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:31.341724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.342417 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.342418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:31.342992 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:31.342995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.343322 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.343323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:31.343361 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:31.343787 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:31.343791 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:31.345451 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":143,"threads_started":1}
W20260809 00:33:31.345635 30904 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:31.341133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:45621 (local address 127.21.41.254:46763)
0809 00:33:31.341264 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:31.341266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:31.341330 (+ 64us) server_negotiation.cc:407] Connection header received
0809 00:33:31.341395 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:31.341396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:31.341406 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:31.341454 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:31.341800 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.342295 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.343066 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.343259 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.343309 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:31.345554 (+ 2245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:45621: BlockingRecv error: recv got EOF from 127.21.41.193:45621 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
Aug 09 00:33:31 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:31 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:31.473749 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:31.468696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:46569)
0809 00:33:31.468888 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:31.468899 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:31.468958 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:31.469115 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:31.469115 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:31.469289 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:31.469348 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:31.469349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.470016 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.470017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:31.470681 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:31.470685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.471246 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.471247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:31.471323 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:31.471798 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:31.471803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:31.473650 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":127,"threads_started":1}
W20260809 00:33:31.473796 30904 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:31.468822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:46569 (local address 127.21.41.254:46763)
0809 00:33:31.468890 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:31.468892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:31.468959 (+ 67us) server_negotiation.cc:407] Connection header received
0809 00:33:31.469024 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:31.469025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:31.469036 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:31.469079 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:31.469421 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.469932 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.470776 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.471127 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.471196 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:31.473734 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:46569: BlockingRecv error: recv got EOF from 127.21.41.194:46569 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 00:33:31 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:31 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:31.610416 30907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:31.605702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:38787)
0809 00:33:31.605880 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:31.605892 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:31.605966 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:31.606097 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:31.606098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:31.606291 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:31.606371 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:31.606373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.606992 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.606993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:31.607530 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:31.607531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.607879 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.607879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:31.607922 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:31.608407 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:31.608413 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:31.610295 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
W20260809 00:33:31.610453 30904 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:31.605816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:38787 (local address 127.21.41.254:46763)
0809 00:33:31.605872 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:31.605874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:31.605968 (+ 94us) server_negotiation.cc:407] Connection header received
0809 00:33:31.606012 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:31.606013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:31.606025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:31.606073 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:31.606423 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.606921 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.607605 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:31.607803 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:31.607863 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:31.610392 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:38787: BlockingRecv error: recv got EOF from 127.21.41.195:38787 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 00:33:32 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:32 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:32.350919 30908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:32.346134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:47853)
0809 00:33:32.346273 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:32.346286 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:32.346410 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:32.346555 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:32.346556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:32.346782 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:32.346835 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:32.346837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.347474 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.347475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:32.348038 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:32.348040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.348364 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.348365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:32.348406 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:32.348930 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:32.348935 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:32.350857 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":58,"threads_started":1}
W20260809 00:33:32.351065 30909 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:32.346232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:47853 (local address 127.21.41.254:46763)
0809 00:33:32.346396 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:32.346398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:32.346407 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:32.346467 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:32.346467 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:32.346477 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:32.346533 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:32.346907 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.347406 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.348111 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.348300 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.348348 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:32.350976 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:47853: BlockingRecv error: recv got EOF from 127.21.41.193:47853 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
Aug 09 00:33:32 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:32 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:32.479169 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:32.474406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:32781)
0809 00:33:32.474572 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:32.474585 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:32.474656 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:32.474819 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:32.474820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:32.475058 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:32.475108 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:32.475110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.475805 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.475807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:32.476368 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:32.476370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.476701 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.476702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:32.476750 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:32.477228 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:32.477233 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:32.479100 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
W20260809 00:33:32.479223 30909 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:32.474495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:32781 (local address 127.21.41.254:46763)
0809 00:33:32.474540 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:32.474542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:32.474660 (+ 118us) server_negotiation.cc:407] Connection header received
0809 00:33:32.474727 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:32.474728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:32.474740 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:32.474798 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:32.475203 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.475708 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.476449 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.476628 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.476678 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:32.479173 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:32781: BlockingRecv error: recv got EOF from 127.21.41.194:32781 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 00:33:32 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:32 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:32.615690 30911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:32.610982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:54237)
0809 00:33:32.611140 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:32.611151 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:32.611219 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:32.611354 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:32.611355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:32.611528 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:32.611577 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:32.611579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.612226 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.612227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:32.612809 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:32.612811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.613126 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.613126 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:32.613166 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:32.613784 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:32.613789 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:32.615581 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
W20260809 00:33:32.615727 30909 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:32.611025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:54237 (local address 127.21.41.254:46763)
0809 00:33:32.611094 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:32.611100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:32.611221 (+ 121us) server_negotiation.cc:407] Connection header received
0809 00:33:32.611276 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:32.611277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:32.611291 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:32.611333 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:32.611653 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.612146 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.612886 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:32.613062 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:32.613111 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:32.615660 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:54237: BlockingRecv error: recv got EOF from 127.21.41.195:54237 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 00:33:33 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:33 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:33.356040 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:33.351400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:51287)
0809 00:33:33.351557 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:33.351570 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:33.351711 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:33.351851 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:33.351851 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:33.352038 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:33.352091 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:33.352093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.352755 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.352756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:33.353371 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:33.353373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.353693 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.353694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:33.353740 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:33.354192 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:33.354197 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:33.355967 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
W20260809 00:33:33.356159 30913 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:33.351517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:51287 (local address 127.21.41.254:46763)
0809 00:33:33.351648 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:33.351650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:33.351691 (+ 41us) server_negotiation.cc:407] Connection header received
0809 00:33:33.351758 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:33.351758 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:33.351768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:33.351831 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:33.352170 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.352680 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.353444 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.353626 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.353674 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:33.356054 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:51287: BlockingRecv error: recv got EOF from 127.21.41.193:51287 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
Aug 09 00:33:33 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:33 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:33.485188 30914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:33.479846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:48895)
0809 00:33:33.480011 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:33.480023 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:33.480095 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:33.480285 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:33.480285 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:33.480457 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:33.480509 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:33.480510 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.481226 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.481227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:33.481790 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:33.481792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.482197 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.482198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:33.482250 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:33.482974 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:33.482979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:33.485126 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
W20260809 00:33:33.485337 30913 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:33.480087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:48895 (local address 127.21.41.254:46763)
0809 00:33:33.480143 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:33.480145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:33.480162 (+ 17us) server_negotiation.cc:407] Connection header received
0809 00:33:33.480190 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:33.480190 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:33.480201 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:33.480266 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:33.480581 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.481132 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.481879 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.482091 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.482170 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:33.485242 (+ 3072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:48895: BlockingRecv error: recv got EOF from 127.21.41.194:48895 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 00:33:33 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:33 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.195@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:33.621615 30915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:33.616301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.195:36311)
0809 00:33:33.616604 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:33.616616 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:33.616712 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:33.616891 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:33.616892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:33.617082 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:33.617129 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:33.617130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.617876 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.617877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:33.618810 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:33.618812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.619192 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.619193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:33.619241 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:33.619687 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:33.619694 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:33.621515 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":107,"threads_started":1}
W20260809 00:33:33.621649 30913 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:33.616416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:36311 (local address 127.21.41.254:46763)
0809 00:33:33.616601 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:33.616603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:33.616692 (+ 89us) server_negotiation.cc:407] Connection header received
0809 00:33:33.616802 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:33.616803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:33.616817 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:33.616884 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:33.617210 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.617751 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.618887 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:33.619092 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:33.619150 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:33.621598 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.195:36311: BlockingRecv error: recv got EOF from 127.21.41.195:36311 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
Aug 09 00:33:34 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:34 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.193@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:34.366040 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:34.356677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.193:60545)
0809 00:33:34.356868 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:34.356880 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:34.356935 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:34.357059 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:34.357060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:34.357223 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:34.357272 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:34.357274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.357967 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.357967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:34.358724 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:34.358727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.359287 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.359288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:34.359341 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:34.364202 (+ 4861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:34.364207 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:34.365980 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":103,"threads_started":1}
W20260809 00:33:34.366261 30917 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:34.356805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:60545 (local address 127.21.41.254:46763)
0809 00:33:34.356934 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:34.356936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:34.356946 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:34.356974 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:34.356974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:34.356984 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:34.357045 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:34.357355 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.357896 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.358863 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.359190 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.359281 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:34.366171 (+ 6890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.193:60545: BlockingRecv error: recv got EOF from 127.21.41.193:60545 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":55,"threads_started":1}
Aug 09 00:33:34 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:33:34 dist-test-slave-zpfg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.41.194@KRBTEST.COM for oryx/127.21.41.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:33:34.491500 30918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:34.485826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:46763 (local address 127.21.41.194:36281)
0809 00:33:34.486040 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:34.486052 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:34.486144 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:34.486365 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:34.486366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:34.486541 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:34.486588 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:34.486590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.487333 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.487334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:34.487910 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:34.487912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.488409 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.488410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:34.488456 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:34.488945 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:34.488950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:34.491375 (+ 2425us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.41.254:46763: Server oryx/127.21.41.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":137,"threads_started":1}
W20260809 00:33:34.491536 30917 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:34.485948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:36281 (local address 127.21.41.254:46763)
0809 00:33:34.486030 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:34.486032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:34.486125 (+ 93us) server_negotiation.cc:407] Connection header received
0809 00:33:34.486234 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:34.486235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:34.486254 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:34.486327 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:34.486676 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.487226 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.488038 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.488343 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.488404 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:34.491464 (+ 3060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.41.194:36281: BlockingRecv error: recv got EOF from 127.21.41.194:36281 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 00:33:34.532476 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 30441
I20260809 00:33:34.537505 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 30593
I20260809 00:33:34.542497 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 30727
I20260809 00:33:34.546929 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 30371
2026-08-09T00:33:34Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17483 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T00:33:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:34Z Disabled control of system clock
I20260809 00:33:34.575532 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45211
--webserver_interface=127.21.41.254
--webserver_port=0
--builtin_ntp_servers=127.21.41.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:45211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 00:33:34.663378 30922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:34.663595 30922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:34.663615 30922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:34.664824 30922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:34.664870 30922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:34.664882 30922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:34.664896 30922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:34.664956 30922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:34.666285 30922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:45211
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:34.666568 30922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:34.666885 30922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:34.672317 30928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:34.672324 30930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:34.672310 30927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:34.672415 30922 server_base.cc:1034] running on GCE node
I20260809 00:33:34.672848 30922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:34.673255 30922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:34.674443 30922 hybrid_clock.cc:648] HybridClock initialized: now 1786235614674417 us; error 26 us; skew 500 ppm
I20260809 00:33:34.676014 30922 webserver.cc:460] Webserver started at http://127.21.41.254:34537/ using document root <none> and password file <none>
I20260809 00:33:34.676298 30922 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:34.676353 30922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:34.676551 30922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:34.677770 30922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "5ab3cecda999435daf35f9bf8af10326"
format_stamp: "Formatted at 2026-08-09 00:33:34 on dist-test-slave-zpfg"
server_key: "a79bdc4a9a9042de74de0f890335c664"
server_key_iv: "e4c721c44e113107035e406d6bbd7355"
server_key_version: "encryptionkey@0"
I20260809 00:33:34.678181 30922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "5ab3cecda999435daf35f9bf8af10326"
format_stamp: "Formatted at 2026-08-09 00:33:34 on dist-test-slave-zpfg"
server_key: "a79bdc4a9a9042de74de0f890335c664"
server_key_iv: "e4c721c44e113107035e406d6bbd7355"
server_key_version: "encryptionkey@0"
I20260809 00:33:34.679682 30922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:34.680528 30936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:34.680703 30922 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:34.680809 30922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "5ab3cecda999435daf35f9bf8af10326"
format_stamp: "Formatted at 2026-08-09 00:33:34 on dist-test-slave-zpfg"
server_key: "a79bdc4a9a9042de74de0f890335c664"
server_key_iv: "e4c721c44e113107035e406d6bbd7355"
server_key_version: "encryptionkey@0"
I20260809 00:33:34.680891 30922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:34.717214 30922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:34.717588 30922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:34.717763 30922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:34.722067 30922 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:45211
I20260809 00:33:34.722108 30988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:45211 every 8 connection(s)
I20260809 00:33:34.722486 30922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:34.723196 30989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:34.725730 30989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326: Bootstrap starting.
I20260809 00:33:34.726270 30989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:34.726605 30989 log.cc:824] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:34.727270 30989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326: No bootstrap required, opened a new log
I20260809 00:33:34.728765 30989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab3cecda999435daf35f9bf8af10326" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45211 } }
I20260809 00:33:34.728901 30989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:34.728924 30989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab3cecda999435daf35f9bf8af10326, State: Initialized, Role: FOLLOWER
I20260809 00:33:34.729027 30989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab3cecda999435daf35f9bf8af10326" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45211 } }
I20260809 00:33:34.729111 30989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:34.729159 30989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:34.729233 30989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:34.729760 30989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab3cecda999435daf35f9bf8af10326" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45211 } }
I20260809 00:33:34.729878 30989 leader_election.cc:302] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ab3cecda999435daf35f9bf8af10326; no voters:
I20260809 00:33:34.730049 30989 leader_election.cc:288] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:34.730108 30992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:34.730412 30989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:34.730424 30992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [term 1 LEADER]: Becoming Leader. State: Replica: 5ab3cecda999435daf35f9bf8af10326, State: Running, Role: LEADER
I20260809 00:33:34.730559 30992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab3cecda999435daf35f9bf8af10326" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45211 } }
I20260809 00:33:34.730891 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 30922
I20260809 00:33:34.730995 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:34.731007 30994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ab3cecda999435daf35f9bf8af10326. Latest consensus state: current_term: 1 leader_uuid: "5ab3cecda999435daf35f9bf8af10326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab3cecda999435daf35f9bf8af10326" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45211 } } }
I20260809 00:33:34.731091 30994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:34.731161 30993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ab3cecda999435daf35f9bf8af10326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab3cecda999435daf35f9bf8af10326" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45211 } } }
I20260809 00:33:34.731220 21671 external_mini_cluster.cc:1428] Setting key 8db1f660b0ba68f45ef425a3291fec4e
I20260809 00:33:34.731226 30993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:34.731343 31001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:34.732055 31001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:34.733946 31001 catalog_manager.cc:1347] Generated new cluster ID: 6879a21556ad4da3b00f1d62e0b006a5
I20260809 00:33:34.734007 31001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:33:34.737396 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:34.731905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41974 (local address 127.21.41.254:45211)
0809 00:33:34.732552 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:34.732560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:34.732584 (+ 24us) server_negotiation.cc:407] Connection header received
0809 00:33:34.732936 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:34.732962 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:34.733038 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:34.733179 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:34.733481 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.734625 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.735452 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.735886 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.736016 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:34.736735 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:34.736739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:34.736739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:34.736916 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:34.736947 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:34.736958 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:34.737029 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:34.737103 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 00:33:34.737183 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":66,"threads_started":1}
I20260809 00:33:34.751935 31001 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:34.752462 31001 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:34.756531 31001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ab3cecda999435daf35f9bf8af10326: Generated new TSK 0
I20260809 00:33:34.756716 31001 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:33:34.757874 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45211
--builtin_ntp_servers=127.21.41.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 00:33:34.884663 31013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:34.885033 31013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:34.885087 31013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:34.886610 31013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:34.886655 31013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:34.886684 31013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:33:34.888217 31013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:34.888504 31013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:34.888849 31013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:34.895001 31018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:34.895210 31013 server_base.cc:1034] running on GCE node
W20260809 00:33:34.895031 31019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:34.895328 31021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:34.895591 31013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:34.895906 31013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:34.897102 31013 hybrid_clock.cc:648] HybridClock initialized: now 1786235614897070 us; error 47 us; skew 500 ppm
I20260809 00:33:34.898648 31013 webserver.cc:460] Webserver started at http://127.21.41.193:39151/ using document root <none> and password file <none>
I20260809 00:33:34.898939 31013 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:34.899009 31013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:34.899143 31013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:34.900556 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "0950eb3199ed4b779a2ca527e02917ae"
format_stamp: "Formatted at 2026-08-09 00:33:34 on dist-test-slave-zpfg"
server_key: "65935b45b892af1a5b3b9b1f6abbac12"
server_key_iv: "bd65abc0228276b84ff030547764936f"
server_key_version: "encryptionkey@0"
I20260809 00:33:34.900985 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "0950eb3199ed4b779a2ca527e02917ae"
format_stamp: "Formatted at 2026-08-09 00:33:34 on dist-test-slave-zpfg"
server_key: "65935b45b892af1a5b3b9b1f6abbac12"
server_key_iv: "bd65abc0228276b84ff030547764936f"
server_key_version: "encryptionkey@0"
I20260809 00:33:34.902735 31013 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 00:33:34.903738 31027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:34.903970 31013 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:34.904152 31013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "0950eb3199ed4b779a2ca527e02917ae"
format_stamp: "Formatted at 2026-08-09 00:33:34 on dist-test-slave-zpfg"
server_key: "65935b45b892af1a5b3b9b1f6abbac12"
server_key_iv: "bd65abc0228276b84ff030547764936f"
server_key_version: "encryptionkey@0"
I20260809 00:33:34.904242 31013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:34.925812 31013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:34.926237 31013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:34.926484 31013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:34.926759 31013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:34.927124 31013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:34.927165 31013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:34.927196 31013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:34.927213 31013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:34.934561 31013 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:39263
I20260809 00:33:34.934580 31140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:39263 every 8 connection(s)
I20260809 00:33:34.935000 31013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:33:34.940400 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:34.935833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:55451 (local address 127.21.41.254:45211)
0809 00:33:34.935900 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:34.935902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:34.936198 (+ 296us) server_negotiation.cc:407] Connection header received
0809 00:33:34.936708 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:34.936709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:34.936723 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:34.936781 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:34.937479 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.937966 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.939059 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.939267 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.939325 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:34.940147 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:34.940148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:34.940149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:34.940153 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:34.940167 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:34.940170 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:34.940220 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:34.940316 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 00:33:34.940340 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:33:34.940629 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:34.935840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45211 (local address 127.21.41.193:55451)
0809 00:33:34.936071 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:34.936107 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:34.936611 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:34.936864 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:34.936868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:34.936877 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:34.937384 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:34.937389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.938053 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.938054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:34.938976 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:34.938979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:34.939356 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:34.939357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:34.939465 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:34.940009 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:34.940009 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:34.940038 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:34.940061 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:34.940230 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:34.940234 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:34.940236 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:33:34.940293 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:33:34.940375 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
I20260809 00:33:34.941161 31141 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45211
I20260809 00:33:34.941273 31141 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:34.941499 31141 heartbeater.cc:511] Master 127.21.41.254:45211 requested a full tablet report, sending...
I20260809 00:33:34.942116 30953 ts_manager.cc:194] Registered new tserver with Master: 0950eb3199ed4b779a2ca527e02917ae (127.21.41.193:39263)
I20260809 00:33:34.942816 30953 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.193:55451
I20260809 00:33:34.944868 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 31013
I20260809 00:33:34.944940 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:33:34.945158 21671 external_mini_cluster.cc:1428] Setting key 4fb9716f92b885307111b13540918638
I20260809 00:33:34.946171 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45211
--builtin_ntp_servers=127.21.41.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 00:33:35.029616 31144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:35.029939 31144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:35.029982 31144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:35.031395 31144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:35.031464 31144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:35.031505 31144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:33:35.033020 31144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:35.033313 31144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:35.033634 31144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:35.038813 31149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:35.038836 31152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:35.038880 31150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:35.039021 31144 server_base.cc:1034] running on GCE node
I20260809 00:33:35.039294 31144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:35.039496 31144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:35.040655 31144 hybrid_clock.cc:648] HybridClock initialized: now 1786235615040623 us; error 51 us; skew 500 ppm
I20260809 00:33:35.041908 31144 webserver.cc:460] Webserver started at http://127.21.41.194:35055/ using document root <none> and password file <none>
I20260809 00:33:35.042148 31144 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:35.042194 31144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:35.042286 31144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:35.043445 31144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "60310005fa73494e81bc0104eccd082f"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "395f722bd6ab5f17b6a5037a76c13991"
server_key_iv: "d52fffc88f47648742664135c82fc528"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.043745 31144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "60310005fa73494e81bc0104eccd082f"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "395f722bd6ab5f17b6a5037a76c13991"
server_key_iv: "d52fffc88f47648742664135c82fc528"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.045002 31144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:33:35.045724 31158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.045902 31144 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:35.046000 31144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "60310005fa73494e81bc0104eccd082f"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "395f722bd6ab5f17b6a5037a76c13991"
server_key_iv: "d52fffc88f47648742664135c82fc528"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.046103 31144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:35.056942 31144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:35.057291 31144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:35.057466 31144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:35.057717 31144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:35.058071 31144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:35.058130 31144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.058172 31144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:35.058197 31144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.064575 31144 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:42471
I20260809 00:33:35.064620 31271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:42471 every 8 connection(s)
I20260809 00:33:35.064983 31144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:33:35.070420 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:35.065776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:42879 (local address 127.21.41.254:45211)
0809 00:33:35.065847 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:35.065849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:35.066097 (+ 248us) server_negotiation.cc:407] Connection header received
0809 00:33:35.066786 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:35.066788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:35.066801 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:35.066852 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:35.067533 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:35.068062 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:35.069114 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:35.069328 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:35.069393 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:35.070166 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:35.070168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:35.070168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:35.070172 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:35.070184 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:35.070188 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:35.070233 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:35.070319 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 00:33:35.070362 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:33:35.070715 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:35.065760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45211 (local address 127.21.41.194:42879)
0809 00:33:35.065972 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:35.066006 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:35.066612 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:35.066929 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:35.066933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:35.066941 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:35.067438 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:35.067442 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:35.068200 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:35.068201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:35.068998 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:35.069000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:35.069406 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:35.069407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:35.069518 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:35.070039 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:35.070040 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:35.070068 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:35.070090 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:35.070243 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:35.070247 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:35.070249 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:33:35.070301 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:33:35.070442 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260809 00:33:35.071308 31272 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45211
I20260809 00:33:35.071345 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 31144
I20260809 00:33:35.071391 31272 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:35.071455 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:33:35.071656 21671 external_mini_cluster.cc:1428] Setting key 13755801fc81753d9c8f29505ceb13bb
I20260809 00:33:35.071619 31272 heartbeater.cc:511] Master 127.21.41.254:45211 requested a full tablet report, sending...
I20260809 00:33:35.072098 30952 ts_manager.cc:194] Registered new tserver with Master: 60310005fa73494e81bc0104eccd082f (127.21.41.194:42471)
I20260809 00:33:35.072589 30952 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.194:42879
I20260809 00:33:35.072676 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45211
--builtin_ntp_servers=127.21.41.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 00:33:35.155230 31275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:35.155460 31275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:35.155483 31275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:35.156581 31275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:35.156617 31275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:35.156642 31275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:35.158012 31275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:35.158202 31275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:35.158473 31275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:35.163518 31280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:35.163527 31283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:35.163543 31275 server_base.cc:1034] running on GCE node
W20260809 00:33:35.163527 31281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:35.163920 31275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:35.164165 31275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:35.165337 31275 hybrid_clock.cc:648] HybridClock initialized: now 1786235615165325 us; error 32 us; skew 500 ppm
I20260809 00:33:35.166688 31275 webserver.cc:460] Webserver started at http://127.21.41.195:42917/ using document root <none> and password file <none>
I20260809 00:33:35.166921 31275 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:35.166986 31275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:35.167135 31275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:35.168347 31275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "8c0ec7f38cd84f269782cecdeb5891f3"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "73308d64f74cad81b674f3414b4f8b57"
server_key_iv: "4f9785136626b8a881a0103cdc1114b3"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.168712 31275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "8c0ec7f38cd84f269782cecdeb5891f3"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "73308d64f74cad81b674f3414b4f8b57"
server_key_iv: "4f9785136626b8a881a0103cdc1114b3"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.170048 31275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:33:35.170878 31289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.171053 31275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.171151 31275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "8c0ec7f38cd84f269782cecdeb5891f3"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "73308d64f74cad81b674f3414b4f8b57"
server_key_iv: "4f9785136626b8a881a0103cdc1114b3"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.171239 31275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:35.182680 31275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:35.182993 31275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:35.183158 31275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:35.183387 31275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:35.183779 31275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:35.183869 31275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.183900 31275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:35.183918 31275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.190233 31275 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:43537
I20260809 00:33:35.190270 31402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:43537 every 8 connection(s)
I20260809 00:33:35.190665 31275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:33:35.196040 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:35.191455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:32853 (local address 127.21.41.254:45211)
0809 00:33:35.191517 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:35.191521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:35.191776 (+ 255us) server_negotiation.cc:407] Connection header received
0809 00:33:35.192239 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:35.192240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:35.192251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:35.192304 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:35.193057 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:35.193560 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:35.194593 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:35.194796 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:35.194855 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:35.195750 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:35.195751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:35.195751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:35.195756 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:35.195773 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:35.195778 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:35.195834 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:35.195948 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 00:33:35.195977 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:33:35.196175 31405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:35.191459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45211 (local address 127.21.41.195:32853)
0809 00:33:35.191656 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:35.191690 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:35.192160 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:35.192359 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:35.192362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:35.192370 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:35.192974 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:35.192980 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:35.193640 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:35.193641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:35.194515 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:35.194518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:35.194876 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:35.194876 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:35.194985 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:35.195588 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:35.195588 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:35.195624 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:35.195647 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:35.195844 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:35.195849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:35.195851 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:33:35.195905 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 00:33:35.195986 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260809 00:33:35.196681 31403 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45211
I20260809 00:33:35.196761 31403 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:35.196983 31403 heartbeater.cc:511] Master 127.21.41.254:45211 requested a full tablet report, sending...
I20260809 00:33:35.197386 30952 ts_manager.cc:194] Registered new tserver with Master: 8c0ec7f38cd84f269782cecdeb5891f3 (127.21.41.195:43537)
I20260809 00:33:35.197772 30952 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.195:32853
I20260809 00:33:35.198228 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 31275
I20260809 00:33:35.198336 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:33:35.198547 21671 external_mini_cluster.cc:1428] Setting key 591aa74edd6687ab9c5ed96b6165a17d
I20260809 00:33:35.199489 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 00:33:35.201308 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:35.200815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.21.41.254:45211)
0809 00:33:35.200868 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:35.200873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:35.200998 (+ 125us) server_negotiation.cc:407] Connection header received
0809 00:33:35.201039 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:35.201040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:35.201049 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:35.201104 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:35.201213 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41990: BlockingRecv error: recv got EOF from 127.0.0.1:41990 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:33:35.201411 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 31013
I20260809 00:33:35.208312 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 31144
I20260809 00:33:35.213126 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 31275
I20260809 00:33:35.217525 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 30922
2026-08-09T00:33:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (669 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T00:33:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:35Z Disabled control of system clock
I20260809 00:33:35.245009 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:36265
--webserver_interface=127.21.41.254
--webserver_port=0
--builtin_ntp_servers=127.21.41.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:36265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 00:33:35.326125 31414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:35.326407 31414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:35.326457 31414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:35.327595 31414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:35.327672 31414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:35.327708 31414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:35.327736 31414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:35.327765 31414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:35.329099 31414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:36265
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:36265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:35.329388 31414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:35.329710 31414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:35.334872 31422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:35.334914 31414 server_base.cc:1034] running on GCE node
W20260809 00:33:35.334879 31420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:35.334901 31419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:35.335275 31414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:35.335521 31414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:35.336668 31414 hybrid_clock.cc:648] HybridClock initialized: now 1786235615336662 us; error 34 us; skew 500 ppm
I20260809 00:33:35.337860 31414 webserver.cc:460] Webserver started at http://127.21.41.254:36623/ using document root <none> and password file <none>
I20260809 00:33:35.338111 31414 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:35.338157 31414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:35.338330 31414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:35.339411 31414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "c4be6f0f5fa04a40ad6a5e348ec61f17"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "22b7b2ace755b8c607788e9b360e0086"
server_key_iv: "7ebf4789164d1a40eaefc781adc8dffe"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.339754 31414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "c4be6f0f5fa04a40ad6a5e348ec61f17"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "22b7b2ace755b8c607788e9b360e0086"
server_key_iv: "7ebf4789164d1a40eaefc781adc8dffe"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.341005 31414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:33:35.341734 31428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.342028 31414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:35.342118 31414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "c4be6f0f5fa04a40ad6a5e348ec61f17"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "22b7b2ace755b8c607788e9b360e0086"
server_key_iv: "7ebf4789164d1a40eaefc781adc8dffe"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.342195 31414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:35.345643 31414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:35.345911 31414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:35.346053 31414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:35.349993 31414 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:36265
I20260809 00:33:35.350070 31480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:36265 every 8 connection(s)
I20260809 00:33:35.350423 31414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:35.351126 31481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:35.353823 31481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17: Bootstrap starting.
I20260809 00:33:35.354452 31481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:35.354727 31481 log.cc:824] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:35.355324 31481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17: No bootstrap required, opened a new log
I20260809 00:33:35.356853 31481 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be6f0f5fa04a40ad6a5e348ec61f17" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36265 } }
I20260809 00:33:35.356985 31481 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:35.357007 31481 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4be6f0f5fa04a40ad6a5e348ec61f17, State: Initialized, Role: FOLLOWER
I20260809 00:33:35.357105 31481 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be6f0f5fa04a40ad6a5e348ec61f17" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36265 } }
I20260809 00:33:35.357195 31481 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:35.357240 31481 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:35.357293 31481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:35.357817 31481 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be6f0f5fa04a40ad6a5e348ec61f17" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36265 } }
I20260809 00:33:35.357936 31481 leader_election.cc:302] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4be6f0f5fa04a40ad6a5e348ec61f17; no voters:
I20260809 00:33:35.358110 31481 leader_election.cc:288] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:35.358160 31484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:35.358371 31484 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [term 1 LEADER]: Becoming Leader. State: Replica: c4be6f0f5fa04a40ad6a5e348ec61f17, State: Running, Role: LEADER
I20260809 00:33:35.358394 31481 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:35.358458 31484 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be6f0f5fa04a40ad6a5e348ec61f17" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36265 } }
I20260809 00:33:35.358877 31486 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4be6f0f5fa04a40ad6a5e348ec61f17. Latest consensus state: current_term: 1 leader_uuid: "c4be6f0f5fa04a40ad6a5e348ec61f17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be6f0f5fa04a40ad6a5e348ec61f17" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36265 } } }
I20260809 00:33:35.358965 31486 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:35.359273 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 31414
I20260809 00:33:35.359370 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:35.359599 21671 external_mini_cluster.cc:1428] Setting key 089d9886cd7f92ec2d52a4b11c242aac
I20260809 00:33:35.359592 31493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:35.359853 31485 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4be6f0f5fa04a40ad6a5e348ec61f17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4be6f0f5fa04a40ad6a5e348ec61f17" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 36265 } } }
I20260809 00:33:35.359928 31485 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:35.360280 31493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:35.361814 31493 catalog_manager.cc:1347] Generated new cluster ID: 18c7764d05484c38b7a50a54093e42e6
I20260809 00:33:35.361873 31493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:33:35.364037 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:35.360221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.21.41.254:36265)
0809 00:33:35.360451 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:35.360463 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:35.362567 (+ 2104us) server_negotiation.cc:407] Connection header received
0809 00:33:35.362714 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:35.362719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:35.362754 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:35.362894 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:35.363581 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:35.363585 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:35.363585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:35.363589 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:35.363637 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:35.363648 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:35.363720 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:35.363790 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 00:33:35.363849 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260809 00:33:35.369930 31493 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:35.370087 31493 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:35.376343 31493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4be6f0f5fa04a40ad6a5e348ec61f17: Generated new TSK 0
I20260809 00:33:35.376539 31493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:33:35.383500 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36265
--builtin_ntp_servers=127.21.41.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 00:33:35.465147 31505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:35.465379 31505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:35.465417 31505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:35.466650 31505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:35.466706 31505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:35.466747 31505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:33:35.468096 31505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:35.468364 31505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:35.468663 31505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:35.474490 31510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:35.474498 31511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:35.474687 31513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:35.474594 31505 server_base.cc:1034] running on GCE node
I20260809 00:33:35.474926 31505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:35.475193 31505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:35.476342 31505 hybrid_clock.cc:648] HybridClock initialized: now 1786235615476328 us; error 31 us; skew 500 ppm
I20260809 00:33:35.477655 31505 webserver.cc:460] Webserver started at http://127.21.41.193:32965/ using document root <none> and password file <none>
I20260809 00:33:35.477936 31505 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:35.477981 31505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:35.478132 31505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:35.479315 31505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "4e7641212b1d45458e004a9becc6606e"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "c9312a77fd9a653fa6db4b61ebb47253"
server_key_iv: "254bd22bc514524b74545e7de832e069"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.479664 31505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "4e7641212b1d45458e004a9becc6606e"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "c9312a77fd9a653fa6db4b61ebb47253"
server_key_iv: "254bd22bc514524b74545e7de832e069"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.480959 31505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:33:35.481753 31519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.481983 31505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:33:35.482080 31505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "4e7641212b1d45458e004a9becc6606e"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "c9312a77fd9a653fa6db4b61ebb47253"
server_key_iv: "254bd22bc514524b74545e7de832e069"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.482167 31505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:35.485328 31505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:35.485601 31505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:35.485749 31505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:35.485960 31505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:35.486291 31505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:35.486346 31505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.486413 31505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:35.486433 31505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.491935 31505 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:43307
I20260809 00:33:35.491998 31632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:43307 every 8 connection(s)
I20260809 00:33:35.492336 31505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:33:35.495594 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:35.493135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:33847 (local address 127.21.41.254:36265)
0809 00:33:35.493190 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:35.493192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:35.493413 (+ 221us) server_negotiation.cc:407] Connection header received
0809 00:33:35.493961 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:35.493963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:35.493969 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:35.494031 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:35.495304 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:35.495305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:35.495305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:35.495310 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:35.495326 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:35.495330 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:35.495385 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:35.495508 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 00:33:35.495523 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:33:35.495827 31635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:35.493114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:36265 (local address 127.21.41.193:33847)
0809 00:33:35.493307 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:35.493341 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:35.493827 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:35.494106 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:35.494110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:35.494118 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:35.495148 (+ 1030us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:35.495149 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:35.495177 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:35.495206 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:35.495416 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:35.495424 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:35.495427 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:35.495507 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 00:33:35.495560 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260809 00:33:35.496351 31633 heartbeater.cc:347] Connected to a master server at 127.21.41.254:36265
I20260809 00:33:35.496431 31633 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:35.496594 31633 heartbeater.cc:511] Master 127.21.41.254:36265 requested a full tablet report, sending...
I20260809 00:33:35.497100 31445 ts_manager.cc:194] Registered new tserver with Master: 4e7641212b1d45458e004a9becc6606e (127.21.41.193:43307)
I20260809 00:33:35.498914 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 31505
I20260809 00:33:35.498991 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:33:35.499181 21671 external_mini_cluster.cc:1428] Setting key e31b005dd7b04f158cf1614bc19e5879
I20260809 00:33:35.500103 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36265
--builtin_ntp_servers=127.21.41.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 00:33:35.580420 31636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:35.580626 31636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:35.580648 31636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:35.581815 31636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:35.581858 31636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:35.581884 31636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:33:35.583257 31636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:35.583518 31636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:35.583827 31636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:35.589258 31641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:35.589273 31642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:35.589262 31644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:35.589506 31636 server_base.cc:1034] running on GCE node
I20260809 00:33:35.589793 31636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:35.590040 31636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:35.591224 31636 hybrid_clock.cc:648] HybridClock initialized: now 1786235615591202 us; error 30 us; skew 500 ppm
I20260809 00:33:35.592871 31636 webserver.cc:460] Webserver started at http://127.21.41.194:35733/ using document root <none> and password file <none>
I20260809 00:33:35.593137 31636 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:35.593194 31636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:35.593345 31636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:35.594516 31636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "bedd8d8bc441457182d8a384b9b8a3d7"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "644fb4f6bdf7508204b5366fa43647eb"
server_key_iv: "f538cc13034833644cfe2ca997ca1a5b"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.594921 31636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "bedd8d8bc441457182d8a384b9b8a3d7"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "644fb4f6bdf7508204b5366fa43647eb"
server_key_iv: "f538cc13034833644cfe2ca997ca1a5b"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.596400 31636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:33:35.597375 31650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.597625 31636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:35.597780 31636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "bedd8d8bc441457182d8a384b9b8a3d7"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "644fb4f6bdf7508204b5366fa43647eb"
server_key_iv: "f538cc13034833644cfe2ca997ca1a5b"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.597862 31636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:35.601326 31636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:35.601590 31636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:35.601732 31636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:35.601934 31636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:35.602245 31636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:35.602298 31636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.602351 31636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:35.602382 31636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.608021 31636 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:34169
I20260809 00:33:35.608100 31763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:34169 every 8 connection(s)
I20260809 00:33:35.608413 31636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:33:35.611579 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:35.609225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:54529 (local address 127.21.41.254:36265)
0809 00:33:35.609304 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:35.609307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:35.609572 (+ 265us) server_negotiation.cc:407] Connection header received
0809 00:33:35.610035 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:35.610036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:35.610041 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:35.610101 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:35.611332 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:35.611334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:35.611334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:35.611338 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:35.611355 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:35.611359 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:35.611414 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:35.611511 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 00:33:35.611524 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:33:35.611770 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:35.609205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:36265 (local address 127.21.41.194:54529)
0809 00:33:35.609425 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:35.609458 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:35.609940 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:35.610157 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:35.610160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:35.610167 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:35.611177 (+ 1010us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:35.611177 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:35.611209 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:35.611239 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:35.611443 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:35.611447 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:35.611449 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:33:35.611507 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:33:35.611557 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":104,"threads_started":1}
I20260809 00:33:35.612335 31764 heartbeater.cc:347] Connected to a master server at 127.21.41.254:36265
I20260809 00:33:35.612442 31764 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:35.612596 31764 heartbeater.cc:511] Master 127.21.41.254:36265 requested a full tablet report, sending...
I20260809 00:33:35.612970 31445 ts_manager.cc:194] Registered new tserver with Master: bedd8d8bc441457182d8a384b9b8a3d7 (127.21.41.194:34169)
I20260809 00:33:35.614941 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 31636
I20260809 00:33:35.615020 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:33:35.615175 21671 external_mini_cluster.cc:1428] Setting key 4e659edc97dd7aa82e9f1c458e1c6dc1
I20260809 00:33:35.616176 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36265
--builtin_ntp_servers=127.21.41.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 00:33:35.697018 31767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:35.697227 31767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:35.697248 31767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:35.698501 31767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:35.698541 31767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:35.698568 31767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:35.699932 31767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:36265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:35.700178 31767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:35.700524 31767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:35.705785 31775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:35.705803 31772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:35.705854 31767 server_base.cc:1034] running on GCE node
W20260809 00:33:35.705803 31773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:35.706255 31767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:35.706525 31767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:35.707666 31767 hybrid_clock.cc:648] HybridClock initialized: now 1786235615707650 us; error 36 us; skew 500 ppm
I20260809 00:33:35.708966 31767 webserver.cc:460] Webserver started at http://127.21.41.195:35337/ using document root <none> and password file <none>
I20260809 00:33:35.709220 31767 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:35.709270 31767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:35.709446 31767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:35.710597 31767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "1292c4cf34e64650944f80af2854e4dd"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "c18a0f47ea08c01ab258612625bcdd2a"
server_key_iv: "7456a71bcf3f3fee3218c823d30b500e"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.710951 31767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "1292c4cf34e64650944f80af2854e4dd"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "c18a0f47ea08c01ab258612625bcdd2a"
server_key_iv: "7456a71bcf3f3fee3218c823d30b500e"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.712261 31767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:33:35.712998 31781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.713173 31767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:35.713274 31767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "1292c4cf34e64650944f80af2854e4dd"
format_stamp: "Formatted at 2026-08-09 00:33:35 on dist-test-slave-zpfg"
server_key: "c18a0f47ea08c01ab258612625bcdd2a"
server_key_iv: "7456a71bcf3f3fee3218c823d30b500e"
server_key_version: "encryptionkey@0"
I20260809 00:33:35.713358 31767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:35.716761 31767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:35.717017 31767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:35.717159 31767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:35.717365 31767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:35.717669 31767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:35.717722 31767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.717760 31767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:35.717789 31767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:35.723879 31767 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:44973
I20260809 00:33:35.723938 31894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:44973 every 8 connection(s)
I20260809 00:33:35.724282 31767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:33:35.727504 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:35.725022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:57899 (local address 127.21.41.254:36265)
0809 00:33:35.725068 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:35.725071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:35.725291 (+ 220us) server_negotiation.cc:407] Connection header received
0809 00:33:35.725877 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:35.725878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:35.725885 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:35.725950 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:35.727243 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:35.727244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:35.727244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:33:35.727249 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:35.727266 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:33:35.727270 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:35.727331 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:35.727416 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 00:33:35.727432 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:33:35.727705 31897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:35.724981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:36265 (local address 127.21.41.195:57899)
0809 00:33:35.725155 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:35.725188 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:35.725756 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:35.726025 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:35.726028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:35.726039 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:35.727078 (+ 1039us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:33:35.727078 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:35.727107 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:33:35.727137 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:35.727331 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:35.727336 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:35.727339 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:33:35.727416 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:33:35.727478 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260809 00:33:35.728228 31895 heartbeater.cc:347] Connected to a master server at 127.21.41.254:36265
I20260809 00:33:35.728304 31895 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:35.728408 31895 heartbeater.cc:511] Master 127.21.41.254:36265 requested a full tablet report, sending...
I20260809 00:33:35.728749 31445 ts_manager.cc:194] Registered new tserver with Master: 1292c4cf34e64650944f80af2854e4dd (127.21.41.195:44973)
I20260809 00:33:35.731078 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 31767
I20260809 00:33:35.731176 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:33:35.731364 21671 external_mini_cluster.cc:1428] Setting key eba0256dc022ea3098724b0c0f96f700
I20260809 00:33:35.732291 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 00:33:35.734247 31502 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:33:35.733753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45728 (local address 127.21.41.254:36265)
0809 00:33:35.733930 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:35.733932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:35.733940 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:33:35.733968 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:35.733969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:35.733973 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:35.734020 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:35.734183 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45728: BlockingRecv error: recv got EOF from 127.0.0.1:45728 (error 108)
Metrics: {"server-negotiator.queue_time_us":152}
I20260809 00:33:35.734360 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 31505
I20260809 00:33:35.740046 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 31636
I20260809 00:33:35.744642 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 31767
I20260809 00:33:35.749039 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 31414
2026-08-09T00:33:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (531 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:33:35 dist-test-slave-zpfg krb5kdc[31905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:33:35 dist-test-slave-zpfg krb5kdc[31905](info): set up 2 sockets
Aug 09 00:33:35 dist-test-slave-zpfg krb5kdc[31905](info): commencing operation
krb5kdc: starting...
W20260809 00:33:36.844997 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:33:36 dist-test-slave-zpfg krb5kdc[31905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:33:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:33:36.965847 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42775
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:42775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:37.049027 31918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:37.049283 31918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:37.049321 31918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:37.050495 31918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:33:37.050565 31918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:37.050590 31918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:37.050612 31918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:33:37.050640 31918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:33:37.052145 31918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:42775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:37.052423 31918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:37.052651 31918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:37.057618 31924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:37.057605 31923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:37.057600 31926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:37.057925 31918 server_base.cc:1034] running on GCE node
I20260809 00:33:37.058220 31918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:37.058547 31918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:37.059717 31918 hybrid_clock.cc:648] HybridClock initialized: now 1786235617059705 us; error 34 us; skew 500 ppm
Aug 09 00:33:37 dist-test-slave-zpfg krb5kdc[31905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:37.062254 31918 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:33:37.062768 31918 webserver.cc:460] Webserver started at http://127.21.41.254:44837/ using document root <none> and password file <none>
I20260809 00:33:37.063004 31918 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:37.063074 31918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:37.063226 31918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:37.064306 31918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "4947cdda148f48cba09c213c3efd8038"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "58f498d750cc46ee115f5ff394d0a9dc"
server_key_iv: "abe13e81e44c1b12b7336bcdf74584b7"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.064663 31918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "4947cdda148f48cba09c213c3efd8038"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "58f498d750cc46ee115f5ff394d0a9dc"
server_key_iv: "abe13e81e44c1b12b7336bcdf74584b7"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.066006 31918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:33:37.066926 31933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:37.067121 31918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:33:37.067224 31918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "4947cdda148f48cba09c213c3efd8038"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "58f498d750cc46ee115f5ff394d0a9dc"
server_key_iv: "abe13e81e44c1b12b7336bcdf74584b7"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.067299 31918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:37.078727 31918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:37.080720 31918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:37.080914 31918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:37.085049 31918 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:42775
I20260809 00:33:37.085090 31985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:42775 every 8 connection(s)
I20260809 00:33:37.085457 31918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:33:37.086549 31986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:37.088871 31986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038: Bootstrap starting.
I20260809 00:33:37.089406 31986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:37.089641 31986 log.cc:824] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:37.090457 31986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038: No bootstrap required, opened a new log
I20260809 00:33:37.090904 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 31918
I20260809 00:33:37.091020 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:33:37.091245 21671 external_mini_cluster.cc:1428] Setting key 72deb2fd7ae66cc43b7575d9befa83f6
I20260809 00:33:37.092929 31986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4947cdda148f48cba09c213c3efd8038" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42775 } }
I20260809 00:33:37.093125 31986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:37.093171 31986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4947cdda148f48cba09c213c3efd8038, State: Initialized, Role: FOLLOWER
I20260809 00:33:37.093333 31986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4947cdda148f48cba09c213c3efd8038" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42775 } }
I20260809 00:33:37.093432 31986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:33:37.093470 31986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:33:37.093529 31986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:37.094689 31986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4947cdda148f48cba09c213c3efd8038" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42775 } }
I20260809 00:33:37.094825 31986 leader_election.cc:302] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4947cdda148f48cba09c213c3efd8038; no voters:
I20260809 00:33:37.095042 31986 leader_election.cc:288] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:33:37.095067 31991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:37.095196 31991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [term 1 LEADER]: Becoming Leader. State: Replica: 4947cdda148f48cba09c213c3efd8038, State: Running, Role: LEADER
I20260809 00:33:37.095295 31991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4947cdda148f48cba09c213c3efd8038" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42775 } }
I20260809 00:33:37.095338 31986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:33:37.095662 31991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4947cdda148f48cba09c213c3efd8038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4947cdda148f48cba09c213c3efd8038" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42775 } } }
I20260809 00:33:37.095742 31991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:37.095889 31991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4947cdda148f48cba09c213c3efd8038. Latest consensus state: current_term: 1 leader_uuid: "4947cdda148f48cba09c213c3efd8038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4947cdda148f48cba09c213c3efd8038" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42775 } } }
I20260809 00:33:37.095968 31991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038 [sys.catalog]: This master's current role is: LEADER
I20260809 00:33:37.096454 31996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:33:37.097021 31996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:33:37.098975 31996 catalog_manager.cc:1347] Generated new cluster ID: eeb45ef8595844beb26ab7a3c73ff384
I20260809 00:33:37.099025 31996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:33:37 dist-test-slave-zpfg krb5kdc[31905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:37.103873 31990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.091859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39094 (local address 127.21.41.254:42775)
0809 00:33:37.092066 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:37.092073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:37.092125 (+ 52us) server_negotiation.cc:407] Connection header received
0809 00:33:37.092912 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:37.092920 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:37.092973 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:37.093089 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:37.094001 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.095062 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.096184 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.096800 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.097486 (+ 686us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.099901 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:37.099905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:37.099905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:37.099917 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:37.101923 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:37.102296 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:37.102299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:37.102299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:37.102441 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:37.102635 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:37.102635 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:37.102636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:37.102920 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:37.103458 (+ 538us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:37.103524 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 00:33:37.103660 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
I20260809 00:33:37.111981 31996 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:33:37.112500 31996 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:33:37.116624 31996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038: Generated new TSK 0
I20260809 00:33:37.116825 31996 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:33:37.149066 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42775
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:37.233249 32012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:37.233455 32012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:37.233479 32012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:37.234812 32012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:37.234858 32012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:37.234884 32012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:33:37.236304 32012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:37.236493 32012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:37.236850 32012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:37.242367 32017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:37.242424 32018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:37.242375 32020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:37.242597 32012 server_base.cc:1034] running on GCE node
I20260809 00:33:37.242905 32012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:37.243221 32012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:37.244366 32012 hybrid_clock.cc:648] HybridClock initialized: now 1786235617244345 us; error 39 us; skew 500 ppm
Aug 09 00:33:37 dist-test-slave-zpfg krb5kdc[31905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:37.246893 32012 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:33:37.247321 32012 webserver.cc:460] Webserver started at http://127.21.41.193:38549/ using document root <none> and password file <none>
I20260809 00:33:37.247566 32012 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:37.247637 32012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:37.247784 32012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:37.248818 32012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "5ea576b7da6d4be2954d1c8456f44271"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "f82f84c22b0a21035eef31daef593e59"
server_key_iv: "917c070fba01da6685549cd4cad1c71e"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.249167 32012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "5ea576b7da6d4be2954d1c8456f44271"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "f82f84c22b0a21035eef31daef593e59"
server_key_iv: "917c070fba01da6685549cd4cad1c71e"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.250512 32012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:33:37.251233 32027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:37.251410 32012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:37.251505 32012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "5ea576b7da6d4be2954d1c8456f44271"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "f82f84c22b0a21035eef31daef593e59"
server_key_iv: "917c070fba01da6685549cd4cad1c71e"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.251587 32012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:37.268105 32012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:37.270452 32012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:37.270609 32012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:37.270846 32012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:37.271174 32012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:37.271229 32012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:37.271270 32012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:37.271306 32012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:37.277199 32012 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:44445
I20260809 00:33:37.277246 32140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:44445 every 8 connection(s)
I20260809 00:33:37.277570 32012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:33:37 dist-test-slave-zpfg krb5kdc[31905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:37.284626 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 32012
I20260809 00:33:37.284750 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:33:37.284961 21671 external_mini_cluster.cc:1428] Setting key d205aee801200b2974c51bf0c5731473
I20260809 00:33:37.287454 31990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.278308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:46849 (local address 127.21.41.254:42775)
0809 00:33:37.278379 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:37.278382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:37.278642 (+ 260us) server_negotiation.cc:407] Connection header received
0809 00:33:37.279101 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:37.279102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:37.279115 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:37.279181 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:37.280123 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.280628 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.281587 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.281820 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.281879 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.284157 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:37.284158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:37.284159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:37.284168 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:37.285850 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:37.286256 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:37.286258 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:37.286258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:37.286321 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:37.286672 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:37.286672 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:37.286673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:37.286792 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:37.286843 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:37.287209 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 00:33:37.287328 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:33:37.287570 32143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.278307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42775 (local address 127.21.41.193:46849)
0809 00:33:37.278516 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:37.278552 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:37.279019 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:37.279261 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:37.279264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:37.279848 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:37.280035 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.280040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.280708 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.280708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.281504 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.281506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.281899 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.281900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.282009 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.282547 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:37.282556 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:37.284051 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:37.285932 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:37.285940 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:37.285943 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:37.286177 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:37.286440 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:37.286441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:37.286441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:37.286600 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:37.286857 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:37.286864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:37.287035 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 00:33:37.287162 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 00:33:37.287355 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":122,"threads_started":1}
I20260809 00:33:37.288098 32141 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42775
I20260809 00:33:37.288215 32141 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:37.288398 32141 heartbeater.cc:511] Master 127.21.41.254:42775 requested a full tablet report, sending...
I20260809 00:33:37.288932 31950 ts_manager.cc:194] Registered new tserver with Master: 5ea576b7da6d4be2954d1c8456f44271 (127.21.41.193:44445)
I20260809 00:33:37.289872 31950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:46849
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:33:37.311021 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42775
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:37.394538 32146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:37.394812 32146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:37.394840 32146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:37.396245 32146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:37.396288 32146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:37.396315 32146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:33:37.397847 32146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:37.398056 32146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:37.398244 32146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:37.403437 32151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:37.403618 32154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:37.403728 32146 server_base.cc:1034] running on GCE node
W20260809 00:33:37.403472 32152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:37.404060 32146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:37.404325 32146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:37.405494 32146 hybrid_clock.cc:648] HybridClock initialized: now 1786235617405482 us; error 27 us; skew 500 ppm
Aug 09 00:33:37 dist-test-slave-zpfg krb5kdc[31905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:37.408064 32146 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:33:37.408560 32146 webserver.cc:460] Webserver started at http://127.21.41.194:37645/ using document root <none> and password file <none>
I20260809 00:33:37.408810 32146 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:37.408877 32146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:37.409024 32146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:37.410085 32146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "b20036cd28e04cc5884eeacb9fbff797"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "33d275519f08f7319e6f2700529eaa02"
server_key_iv: "b6293afdd21b964ba149f7b2c7afc7f5"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.410516 32146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "b20036cd28e04cc5884eeacb9fbff797"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "33d275519f08f7319e6f2700529eaa02"
server_key_iv: "b6293afdd21b964ba149f7b2c7afc7f5"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.411849 32146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:33:37.412565 32161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:37.412791 32146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:33:37.412889 32146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "b20036cd28e04cc5884eeacb9fbff797"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "33d275519f08f7319e6f2700529eaa02"
server_key_iv: "b6293afdd21b964ba149f7b2c7afc7f5"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.412969 32146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:37.433768 32146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:37.436030 32146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:37.436195 32146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:37.436443 32146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:37.436764 32146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:37.436825 32146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:37.436861 32146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:37.436888 32146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:37.442868 32146 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:45959
I20260809 00:33:37.442909 32274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:45959 every 8 connection(s)
I20260809 00:33:37.443261 32146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:33:37.446269 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 32146
I20260809 00:33:37.446406 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:33:37.446694 21671 external_mini_cluster.cc:1428] Setting key 19f85f7bb522dd1bb4450d2a78b48028
Aug 09 00:33:37 dist-test-slave-zpfg krb5kdc[31905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:37.452683 31990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.444075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:42799 (local address 127.21.41.254:42775)
0809 00:33:37.444161 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:37.444164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:37.444318 (+ 154us) server_negotiation.cc:407] Connection header received
0809 00:33:37.444826 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:37.444827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:37.444837 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:37.444889 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:37.445819 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.446350 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.447558 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.447759 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.447850 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.449919 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:37.449921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:37.449921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:37.449929 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:37.451278 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:37.451688 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:37.451689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:37.451690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:37.451753 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:37.451999 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:37.452000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:37.452001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:37.452141 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:37.452218 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:37.452500 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 00:33:37.452590 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:33:37.453042 32277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.444023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42775 (local address 127.21.41.194:42799)
0809 00:33:37.444199 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:37.444236 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:37.444712 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:37.444959 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:37.444962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:37.445537 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:37.445731 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.445736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.446453 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.446454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.447457 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.447459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.447860 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.447861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.447992 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.448500 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:37.448507 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:37.449824 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:37.451376 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:37.451382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:37.451384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:37.451610 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:37.451823 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:37.451824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:37.451824 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:37.451930 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:37.452221 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:37.452228 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:37.452365 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 00:33:37.452479 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 00:33:37.452634 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260809 00:33:37.453616 32275 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42775
I20260809 00:33:37.453697 32275 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:37.453900 32275 heartbeater.cc:511] Master 127.21.41.254:42775 requested a full tablet report, sending...
I20260809 00:33:37.454324 31950 ts_manager.cc:194] Registered new tserver with Master: b20036cd28e04cc5884eeacb9fbff797 (127.21.41.194:45959)
I20260809 00:33:37.454723 31950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:42799
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:33:37.473793 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42775
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:33:37.556720 32280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:33:37.556926 32280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:33:37.556948 32280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:33:37.558033 32280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:33:37.558069 32280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:33:37.558094 32280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:33:37.559552 32280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:33:37.559765 32280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:33:37.559953 32280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:33:37.564971 32285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:37.564992 32286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:33:37.564971 32288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:33:37.565234 32280 server_base.cc:1034] running on GCE node
I20260809 00:33:37.565433 32280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:33:37.565621 32280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:33:37.566757 32280 hybrid_clock.cc:648] HybridClock initialized: now 1786235617566744 us; error 30 us; skew 500 ppm
Aug 09 00:33:37 dist-test-slave-zpfg krb5kdc[31905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:33:37.569437 32280 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:33:37.569979 32280 webserver.cc:460] Webserver started at http://127.21.41.195:33241/ using document root <none> and password file <none>
I20260809 00:33:37.570168 32280 fs_manager.cc:359] Metadata directory not provided
I20260809 00:33:37.570210 32280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:33:37.570298 32280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:33:37.571563 32280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "393bafe676e547bc8e24fd980b4ccf3a"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "8f5bd12cc19c6f0049067ccc52ca4740"
server_key_iv: "ea754ac636efee8332c0c0d603b7d014"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.571934 32280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "393bafe676e547bc8e24fd980b4ccf3a"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "8f5bd12cc19c6f0049067ccc52ca4740"
server_key_iv: "ea754ac636efee8332c0c0d603b7d014"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.573228 32280 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:33:37.573993 32295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:37.574157 32280 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:33:37.574257 32280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "393bafe676e547bc8e24fd980b4ccf3a"
format_stamp: "Formatted at 2026-08-09 00:33:37 on dist-test-slave-zpfg"
server_key: "8f5bd12cc19c6f0049067ccc52ca4740"
server_key_iv: "ea754ac636efee8332c0c0d603b7d014"
server_key_version: "encryptionkey@0"
I20260809 00:33:37.574337 32280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:33:37.607726 32280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:33:37.610139 32280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:33:37.610309 32280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:33:37.610589 32280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:33:37.610929 32280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:33:37.610991 32280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:37.611032 32280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:33:37.611060 32280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:33:37.617801 32280 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:35427
I20260809 00:33:37.617841 32408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:35427 every 8 connection(s)
I20260809 00:33:37.618204 32280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:33:37.619304 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 32280
I20260809 00:33:37.619401 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:33:37.619596 21671 external_mini_cluster.cc:1428] Setting key a571fb06ebb6452a632c56e678e06d6a
Aug 09 00:33:37 dist-test-slave-zpfg krb5kdc[31905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:33:37.627815 31990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.618977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:40837 (local address 127.21.41.254:42775)
0809 00:33:37.619026 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:37.619029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:37.619341 (+ 312us) server_negotiation.cc:407] Connection header received
0809 00:33:37.620012 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:37.620014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:37.620026 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:37.620076 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:37.621019 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.621521 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.622581 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.622772 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.622863 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.625016 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:37.625018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:37.625018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:37.625025 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:37.626547 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:37.626937 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:37.626938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:37.626938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:37.626998 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:37.627246 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:37.627247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:37.627247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:37.627365 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:37.627435 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:37.627681 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 00:33:37.627747 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:33:37.628011 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.618972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42775 (local address 127.21.41.195:40837)
0809 00:33:37.619163 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:37.619213 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:37.619893 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:37.620144 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:37.620149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:37.620724 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:33:37.620922 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.620926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.621616 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.621617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.622486 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.622488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.622874 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.622875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.622999 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.623506 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:33:37.623514 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:33:37.624929 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:33:37.626624 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:37.626629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:37.626631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:37.626846 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:37.627070 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:33:37.627071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:33:37.627071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:33:37.627166 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:33:37.627437 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:33:37.627442 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:33:37.627555 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 00:33:37.627639 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:33:37.627757 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260809 00:33:37.628543 32409 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42775
I20260809 00:33:37.628623 32409 heartbeater.cc:464] Registering TS with master...
I20260809 00:33:37.628759 32409 heartbeater.cc:511] Master 127.21.41.254:42775 requested a full tablet report, sending...
I20260809 00:33:37.629092 31950 ts_manager.cc:194] Registered new tserver with Master: 393bafe676e547bc8e24fd980b4ccf3a (127.21.41.195:35427)
I20260809 00:33:37.629422 31950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:40837
I20260809 00:33:37.631202 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:33:37.638917 31990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.632509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.21.41.254:42775)
0809 00:33:37.632584 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:37.632586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:37.632752 (+ 166us) server_negotiation.cc:407] Connection header received
0809 00:33:37.632828 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:37.632829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:37.632838 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:37.632886 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:33:37.633436 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.633916 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.634731 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.634931 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.635015 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.636209 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:33:37.636211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:33:37.636211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:33:37.636218 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:33:37.637621 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:37.637987 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:37.637988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:37.637989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:37.638050 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:33:37.638263 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:33:37.638264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:33:37.638264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:33:37.638485 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:33:37.638589 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:37.638732 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 00:33:37.638812 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:33:37.640700 31950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:33:37.642380 31950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:33:37.648802 32424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.644827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:44445 (local address 127.0.0.1:38562)
0809 00:33:37.644941 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:37.644952 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:37.645001 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:37.645354 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:37.645355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:37.645357 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:37.645414 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.645416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.646473 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.646474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.647805 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.647808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.648543 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.648544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.648594 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.648604 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 00:33:37.648648 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 00:33:37.648711 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260809 00:33:37.648839 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.644748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54766 (local address 127.21.41.194:45959)
0809 00:33:37.645079 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:37.645108 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:37.645121 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:33:37.645164 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:37.645166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:37.645183 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:37.645281 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:37.645488 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.646681 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.647656 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.648394 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.648469 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.648526 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:37.648704 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:33:37.648752 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":86,"threads_started":1}
I20260809 00:33:37.648995 32425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.644997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38562 (local address 127.21.41.193:44445)
0809 00:33:37.645186 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:37.645190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:37.645199 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:33:37.645237 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:37.645239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:37.645254 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:37.645493 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:37.645571 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.646298 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.647918 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.648422 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.648761 (+ 339us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.648822 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:37.648848 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 00:33:37.648890 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
I20260809 00:33:37.649004 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.644612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:35427 (local address 127.0.0.1:47846)
0809 00:33:37.644816 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:37.644859 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:37.644927 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:37.645333 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:37.645333 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:37.645335 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:37.645395 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.645398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.646730 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.646731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.647824 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.647826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.648767 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.648768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.648851 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.648855 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:33:37.648889 (+ 34us) client_negotiation.cc:240] Negotiation successful
0809 00:33:37.648933 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":34,"thread_start_us":48,"threads_started":1}
I20260809 00:33:37.649344 32422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.644712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47846 (local address 127.21.41.195:35427)
0809 00:33:37.645136 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:37.645140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:37.645151 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:33:37.645187 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:37.645189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:37.645212 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:37.645475 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:37.645537 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.646625 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.647969 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.648645 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.649123 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.649194 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:37.649218 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 00:33:37.649254 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":100,"threads_started":1}
I20260809 00:33:37.649518 32421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.644645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:45959 (local address 127.0.0.1:54766)
0809 00:33:37.644916 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:37.644925 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:37.644967 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:37.645308 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:37.645311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:37.645313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:37.645395 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.645397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.646759 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.646759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.647540 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.647542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.648488 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.648489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.648584 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.648600 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 00:33:37.648673 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:33:37.648713 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":13,"thread_start_us":101,"threads_started":1}
I20260809 00:33:37.649778 32075 tablet_service.cc:1467] Processing CreateTablet for tablet cef208951b0c410cb0bb57c5da859864 (DEFAULT_TABLE table=test-table [id=2a40f0c321794db8ad12b97293af4362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:37.649902 32343 tablet_service.cc:1467] Processing CreateTablet for tablet cef208951b0c410cb0bb57c5da859864 (DEFAULT_TABLE table=test-table [id=2a40f0c321794db8ad12b97293af4362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:37.650151 32075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef208951b0c410cb0bb57c5da859864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:37.650233 32343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef208951b0c410cb0bb57c5da859864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:37.651001 32209 tablet_service.cc:1467] Processing CreateTablet for tablet cef208951b0c410cb0bb57c5da859864 (DEFAULT_TABLE table=test-table [id=2a40f0c321794db8ad12b97293af4362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:33:37.651325 32209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef208951b0c410cb0bb57c5da859864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:33:37.653223 32426 tablet_bootstrap.cc:492] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271: Bootstrap starting.
I20260809 00:33:37.653580 32427 tablet_bootstrap.cc:492] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: Bootstrap starting.
I20260809 00:33:37.653983 32426 tablet_bootstrap.cc:654] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:37.654280 32427 tablet_bootstrap.cc:654] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:37.654623 32426 log.cc:824] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:37.654796 32428 tablet_bootstrap.cc:492] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: Bootstrap starting.
I20260809 00:33:37.654887 32427 log.cc:824] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:37.655435 32428 tablet_bootstrap.cc:654] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: Neither blocks nor log segments found. Creating new log.
I20260809 00:33:37.655603 32426 tablet_bootstrap.cc:492] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271: No bootstrap required, opened a new log
I20260809 00:33:37.655681 32426 ts_tablet_manager.cc:1397] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:33:37.655721 32427 tablet_bootstrap.cc:492] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: No bootstrap required, opened a new log
I20260809 00:33:37.655793 32427 ts_tablet_manager.cc:1397] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:33:37.656050 32428 log.cc:824] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: Log is configured to *not* fsync() on all Append() calls
I20260809 00:33:37.656721 32428 tablet_bootstrap.cc:492] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: No bootstrap required, opened a new log
I20260809 00:33:37.656773 32428 ts_tablet_manager.cc:1397] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:33:37.657694 32427 raft_consensus.cc:348] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "393bafe676e547bc8e24fd980b4ccf3a" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35427 } } peers { permanent_uuid: "5ea576b7da6d4be2954d1c8456f44271" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44445 } } peers { permanent_uuid: "b20036cd28e04cc5884eeacb9fbff797" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45959 } }
I20260809 00:33:37.657850 32427 raft_consensus.cc:374] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:37.657892 32427 raft_consensus.cc:729] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393bafe676e547bc8e24fd980b4ccf3a, State: Initialized, Role: FOLLOWER
I20260809 00:33:37.657788 32426 raft_consensus.cc:348] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "393bafe676e547bc8e24fd980b4ccf3a" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35427 } } peers { permanent_uuid: "5ea576b7da6d4be2954d1c8456f44271" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44445 } } peers { permanent_uuid: "b20036cd28e04cc5884eeacb9fbff797" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45959 } }
I20260809 00:33:37.657935 32426 raft_consensus.cc:374] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:37.657977 32426 raft_consensus.cc:729] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ea576b7da6d4be2954d1c8456f44271, State: Initialized, Role: FOLLOWER
I20260809 00:33:37.658003 32427 consensus_queue.cc:260] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "393bafe676e547bc8e24fd980b4ccf3a" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35427 } } peers { permanent_uuid: "5ea576b7da6d4be2954d1c8456f44271" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44445 } } peers { permanent_uuid: "b20036cd28e04cc5884eeacb9fbff797" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45959 } }
I20260809 00:33:37.658094 32426 consensus_queue.cc:260] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "393bafe676e547bc8e24fd980b4ccf3a" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35427 } } peers { permanent_uuid: "5ea576b7da6d4be2954d1c8456f44271" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44445 } } peers { permanent_uuid: "b20036cd28e04cc5884eeacb9fbff797" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45959 } }
I20260809 00:33:37.658324 32427 ts_tablet_manager.cc:1428] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:33:37.658365 32409 heartbeater.cc:503] Master 127.21.41.254:42775 was elected leader, sending a full tablet report...
I20260809 00:33:37.658377 32426 ts_tablet_manager.cc:1428] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:33:37.658514 32141 heartbeater.cc:503] Master 127.21.41.254:42775 was elected leader, sending a full tablet report...
I20260809 00:33:37.658679 32428 raft_consensus.cc:348] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "393bafe676e547bc8e24fd980b4ccf3a" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35427 } } peers { permanent_uuid: "5ea576b7da6d4be2954d1c8456f44271" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44445 } } peers { permanent_uuid: "b20036cd28e04cc5884eeacb9fbff797" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45959 } }
I20260809 00:33:37.658831 32428 raft_consensus.cc:374] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:33:37.658875 32428 raft_consensus.cc:729] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b20036cd28e04cc5884eeacb9fbff797, State: Initialized, Role: FOLLOWER
I20260809 00:33:37.658985 32428 consensus_queue.cc:260] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "393bafe676e547bc8e24fd980b4ccf3a" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35427 } } peers { permanent_uuid: "5ea576b7da6d4be2954d1c8456f44271" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44445 } } peers { permanent_uuid: "b20036cd28e04cc5884eeacb9fbff797" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45959 } }
I20260809 00:33:37.659230 32428 ts_tablet_manager.cc:1428] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:33:37.659312 32275 heartbeater.cc:503] Master 127.21.41.254:42775 was elected leader, sending a full tablet report...
W20260809 00:33:37.693856 32276 tablet.cc:2369] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:33:37.778405 32142 tablet.cc:2369] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:33:37.868865 32410 tablet.cc:2369] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:33:37.954974 32433 raft_consensus.cc:482] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:33:37.955102 32433 raft_consensus.cc:504] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "393bafe676e547bc8e24fd980b4ccf3a" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35427 } } peers { permanent_uuid: "5ea576b7da6d4be2954d1c8456f44271" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44445 } } peers { permanent_uuid: "b20036cd28e04cc5884eeacb9fbff797" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45959 } }
I20260809 00:33:37.955518 32433 leader_election.cc:288] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 393bafe676e547bc8e24fd980b4ccf3a (127.21.41.195:35427), b20036cd28e04cc5884eeacb9fbff797 (127.21.41.194:45959)
I20260809 00:33:37.958648 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.955868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:42355 (local address 127.21.41.194:45959)
0809 00:33:37.955915 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:37.955917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:37.956013 (+ 96us) server_negotiation.cc:407] Connection header received
0809 00:33:37.956043 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:37.956044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:37.956053 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:37.956109 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:37.956331 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.956807 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.957908 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.958241 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.958330 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.958361 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:37.958530 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 00:33:37.958569 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:33:37.958649 32422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.955931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:58065 (local address 127.21.41.195:35427)
0809 00:33:37.955974 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:37.955976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:37.955986 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:33:37.956076 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:37.956077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:37.956086 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:37.956184 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:33:37.956363 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.956806 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.957830 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.958240 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.958334 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.958343 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:37.958533 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 00:33:37.958570 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:33:37.958720 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.955753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:45959 (local address 127.21.41.193:42355)
0809 00:33:37.955890 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:37.955901 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:37.955964 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:37.956138 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:37.956139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:37.956141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:37.956240 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.956242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.956894 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.956895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.957816 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.957819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.958364 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.958365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.958449 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.958454 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:33:37.958525 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 00:33:37.958570 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":65,"thread_start_us":57,"threads_started":1}
I20260809 00:33:37.958734 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:37.955753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:35427 (local address 127.21.41.193:58065)
0809 00:33:37.955881 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:37.955894 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:33:37.955974 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:33:37.956187 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:33:37.956188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:33:37.956189 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:33:37.956269 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.956270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.956900 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.956901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.957736 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:33:37.957737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:33:37.958332 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:33:37.958333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:33:37.958449 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:37.958453 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:33:37.958525 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 00:33:37.958571 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":25,"thread_start_us":63,"threads_started":1}
I20260809 00:33:37.959118 32363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef208951b0c410cb0bb57c5da859864" candidate_uuid: "5ea576b7da6d4be2954d1c8456f44271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393bafe676e547bc8e24fd980b4ccf3a" is_pre_election: true
I20260809 00:33:37.959280 32363 raft_consensus.cc:2454] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ea576b7da6d4be2954d1c8456f44271 in term 0.
I20260809 00:33:37.959264 32229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef208951b0c410cb0bb57c5da859864" candidate_uuid: "5ea576b7da6d4be2954d1c8456f44271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b20036cd28e04cc5884eeacb9fbff797" is_pre_election: true
I20260809 00:33:37.959445 32229 raft_consensus.cc:2454] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ea576b7da6d4be2954d1c8456f44271 in term 0.
I20260809 00:33:37.959699 32029 leader_election.cc:302] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 393bafe676e547bc8e24fd980b4ccf3a, 5ea576b7da6d4be2954d1c8456f44271; no voters:
I20260809 00:33:37.959853 32433 raft_consensus.cc:2781] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:33:37.959913 32433 raft_consensus.cc:482] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:33:37.959935 32433 raft_consensus.cc:3037] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:37.960639 32433 raft_consensus.cc:504] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "393bafe676e547bc8e24fd980b4ccf3a" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35427 } } peers { permanent_uuid: "5ea576b7da6d4be2954d1c8456f44271" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44445 } } peers { permanent_uuid: "b20036cd28e04cc5884eeacb9fbff797" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45959 } }
I20260809 00:33:37.960783 32433 leader_election.cc:288] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [CANDIDATE]: Term 1 election: Requested vote from peers 393bafe676e547bc8e24fd980b4ccf3a (127.21.41.195:35427), b20036cd28e04cc5884eeacb9fbff797 (127.21.41.194:45959)
I20260809 00:33:37.960950 32363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef208951b0c410cb0bb57c5da859864" candidate_uuid: "5ea576b7da6d4be2954d1c8456f44271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393bafe676e547bc8e24fd980b4ccf3a"
I20260809 00:33:37.961019 32363 raft_consensus.cc:3037] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:37.961851 32363 raft_consensus.cc:2454] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ea576b7da6d4be2954d1c8456f44271 in term 1.
I20260809 00:33:37.961992 32229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef208951b0c410cb0bb57c5da859864" candidate_uuid: "5ea576b7da6d4be2954d1c8456f44271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b20036cd28e04cc5884eeacb9fbff797"
I20260809 00:33:37.962019 32029 leader_election.cc:302] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 393bafe676e547bc8e24fd980b4ccf3a, 5ea576b7da6d4be2954d1c8456f44271; no voters:
I20260809 00:33:37.962080 32229 raft_consensus.cc:3037] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:33:37.962116 32433 raft_consensus.cc:2781] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:33:37.962378 32433 raft_consensus.cc:686] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 1 LEADER]: Becoming Leader. State: Replica: 5ea576b7da6d4be2954d1c8456f44271, State: Running, Role: LEADER
I20260809 00:33:37.962484 32433 consensus_queue.cc:237] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "393bafe676e547bc8e24fd980b4ccf3a" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35427 } } peers { permanent_uuid: "5ea576b7da6d4be2954d1c8456f44271" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44445 } } peers { permanent_uuid: "b20036cd28e04cc5884eeacb9fbff797" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45959 } }
I20260809 00:33:37.962981 32229 raft_consensus.cc:2454] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ea576b7da6d4be2954d1c8456f44271 in term 1.
I20260809 00:33:37.963349 31950 catalog_manager.cc:5495] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ea576b7da6d4be2954d1c8456f44271 (127.21.41.193). New cstate: current_term: 1 leader_uuid: "5ea576b7da6d4be2954d1c8456f44271" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "393bafe676e547bc8e24fd980b4ccf3a" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ea576b7da6d4be2954d1c8456f44271" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b20036cd28e04cc5884eeacb9fbff797" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45959 } health_report { overall_health: UNKNOWN } } }
I20260809 00:33:38.002662 32425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:38.000138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.21.41.193:44445)
0809 00:33:38.000211 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 00:33:38.000213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:33:38.000224 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:33:38.000308 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:33:38.000309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:33:38.000328 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:33:38.000409 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:33:38.000554 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:38.001032 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:38.001831 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:33:38.002031 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:33:38.002113 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:33:38.002296 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:33:38.002422 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:33:38.002526 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 00:33:38.002545 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 00:33:38.002567 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:33:38.004859 32363 raft_consensus.cc:1261] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a [term 1 FOLLOWER]: Refusing update from remote peer 5ea576b7da6d4be2954d1c8456f44271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:38.004974 32229 raft_consensus.cc:1261] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797 [term 1 FOLLOWER]: Refusing update from remote peer 5ea576b7da6d4be2954d1c8456f44271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:33:38.005292 32433 consensus_queue.cc:1035] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "393bafe676e547bc8e24fd980b4ccf3a" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 35427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:38.005434 32437 consensus_queue.cc:1035] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b20036cd28e04cc5884eeacb9fbff797" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:33:38.010308 31949 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:38.010448 31949 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:33:38.011273 31949 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4947cdda148f48cba09c213c3efd8038: Sending DeleteTablet for 3 replicas of tablet cef208951b0c410cb0bb57c5da859864
I20260809 00:33:38.011654 32209 tablet_service.cc:1514] Processing DeleteTablet for tablet cef208951b0c410cb0bb57c5da859864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:38 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:54766
I20260809 00:33:38.011675 32075 tablet_service.cc:1514] Processing DeleteTablet for tablet cef208951b0c410cb0bb57c5da859864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:38 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:38562
I20260809 00:33:38.011674 32343 tablet_service.cc:1514] Processing DeleteTablet for tablet cef208951b0c410cb0bb57c5da859864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:33:38 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:47846
I20260809 00:33:38.011904 32451 tablet_replica.cc:330] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: stopping tablet replica
I20260809 00:33:38.012001 32451 raft_consensus.cc:2229] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:38.012094 32450 tablet_replica.cc:330] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271: stopping tablet replica
I20260809 00:33:38.012163 32451 raft_consensus.cc:2258] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:38.012200 32450 raft_consensus.cc:2229] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:33:38.012291 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 32012
I20260809 00:33:38.012336 32450 raft_consensus.cc:2258] T cef208951b0c410cb0bb57c5da859864 P 5ea576b7da6d4be2954d1c8456f44271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:38.013191 32451 ts_tablet_manager.cc:1905] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:38.013922 32449 tablet_replica.cc:330] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: stopping tablet replica
I20260809 00:33:38.013999 32449 raft_consensus.cc:2229] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:33:38.014079 32449 raft_consensus.cc:2258] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:33:38.014514 32449 ts_tablet_manager.cc:1905] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:33:38.015195 32451 ts_tablet_manager.cc:1918] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:38.015352 32451 log.cc:1196] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/cef208951b0c410cb0bb57c5da859864
I20260809 00:33:38.015700 32451 ts_tablet_manager.cc:1939] T cef208951b0c410cb0bb57c5da859864 P 393bafe676e547bc8e24fd980b4ccf3a: Deleting consensus metadata
I20260809 00:33:38.016045 32449 ts_tablet_manager.cc:1918] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:33:38.016119 32449 log.cc:1196] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/cef208951b0c410cb0bb57c5da859864
I20260809 00:33:38.016222 31935 catalog_manager.cc:4841] TS 393bafe676e547bc8e24fd980b4ccf3a (127.21.41.195:35427): tablet cef208951b0c410cb0bb57c5da859864 (table test-table [id=2a40f0c321794db8ad12b97293af4362]) successfully deleted
I20260809 00:33:38.016383 32449 ts_tablet_manager.cc:1939] T cef208951b0c410cb0bb57c5da859864 P b20036cd28e04cc5884eeacb9fbff797: Deleting consensus metadata
I20260809 00:33:38.016778 31937 catalog_manager.cc:4841] TS b20036cd28e04cc5884eeacb9fbff797 (127.21.41.194:45959): tablet cef208951b0c410cb0bb57c5da859864 (table test-table [id=2a40f0c321794db8ad12b97293af4362]) successfully deleted
W20260809 00:33:38.018946 31935 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:44445 (error 108)
I20260809 00:33:38.019193 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 32146
I20260809 00:33:38.019421 32421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:33:38.019300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:44445 (local address 127.0.0.1:38580)
0809 00:33:38.019345 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 00:33:38.019383 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:44445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 00:33:38.020040 31935 catalog_manager.cc:4584] TS 5ea576b7da6d4be2954d1c8456f44271 (127.21.41.193:44445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cef208951b0c410cb0bb57c5da859864: Network error: Client connection negotiation failed: client connection to 127.21.41.193:44445: connect: Connection refused (error 111)
I20260809 00:33:38.024734 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 32280
I20260809 00:33:38.029448 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 31918
2026-08-09T00:33:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2281 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:33:38 dist-test-slave-zpfg krb5kdc[32455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:33:38 dist-test-slave-zpfg krb5kdc[32455](info): set up 2 sockets
Aug 09 00:33:38 dist-test-slave-zpfg krb5kdc[32455](info): commencing operation
krb5kdc: starting...
W20260809 00:33:39.172925 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:33:39 dist-test-slave-zpfg krb5kdc[32455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:33:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:33:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.41.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.41.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/rangeradmin_127.21.41.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/rangeradmin_127.21.41.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.41.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.41.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/rangerlookup_127.21.41.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/rangerlookup_127.21.41.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.41.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.41.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/HTTP_127.21.41.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/HTTP_127.21.41.212@KRBTEST.COM.keytab.
I20260809 00:33:39.348910 21671 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskbJF9WP/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/postgres -l logfile start
2026-08-09 00:33:42.000 UTC [32485] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 00:33:42.000 UTC [32485] LOG: listening on IPv4 address "127.21.41.212", port 57935
2026-08-09 00:33:42.004 UTC [32485] LOG: listening on Unix socket "/tmp/.s.PGSQL.57935"
2026-08-09 00:33:42.040 UTC [32488] LOG: database system was shut down at 2026-08-09 00:33:40 UTC
2026-08-09 00:33:42.044 UTC [32485] LOG: database system is ready to accept connections
I20260809 00:33:43.009235 21671 mini_postgres.cc:96] Postgres bound to 57935
2026-08-09 00:33:43.016 UTC [32496] FATAL: database "slave" does not exist
127.21.41.212:57935 - accepting connections
I20260809 00:33:43.016942 21671 mini_ranger.cc:161] Starting Ranger...
I20260809 00:33:43.032755 21671 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 00:33:43.527100 21671 mini_ranger.cc:87] Created ranger Postgres database
I20260809 00:33:43.527187 21671 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-admin
2026-08-09 00:33:43,826 [I] DB FLAVOR :POSTGRES
2026-08-09 00:33:43,827 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:33:43,827 [I] Checking connection..
2026-08-09 00:33:43,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 00:33:44,072 [I] Checking connection passed.
2026-08-09 00:33:44,072 [I] --------- Verifying version history table ---------
2026-08-09 00:33:44,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:33:44,310 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:33:44,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:33:44,559 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:33:44,559 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 00:33:44,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 00:33:44.792 UTC [32560] WARNING: there is no transaction in progress
2026-08-09 00:33:44,798 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 00:33:44,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:33:45,056 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 00:33:45,056 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:33:45,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 00:33:45,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 00:33:45,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zpfg.c.gcp-upstream.internal','N') ;"
2026-08-09 00:33:45,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:33:46,048 [I] Table x_portal_user does not exist in database ranger
2026-08-09 00:33:46,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:33:46,327 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 00:33:46,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:33:46,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:33:46,811 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 00:33:46,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 00:33:47.838 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:47.855 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:47.872 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:47.887 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:47.903 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:47.925 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:47.984 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:47.991 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:48.009 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:48.021 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:48.033 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:48.049 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:48.060 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:48.068 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:48.076 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:48.084 UTC [32669] WARNING: there is no transaction in progress
2026-08-09 00:33:48,803 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 00:33:48,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 00:33:49,043 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 00:33:49,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:33:49,301 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 00:33:49,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:33:49,554 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 00:33:49,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:33:49,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:33:50,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zpfg.c.gcp-upstream.internal';"
2026-08-09 00:33:50,270 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 00:33:50,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 00:33:50,513 [I] DB_PATCHES have already been applied
I20260809 00:33:50.519107 21671 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-admin:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskbJF9WP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 00:33:50.519186 21671 mini_ranger.cc:193] Using host: 127.21.41.212
I20260809 00:33:50.522799 21671 mini_ranger.cc:227] Ranger admin URL: http://127.21.41.212:59793
Aug 09, 2026 12:33:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:33:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:33:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:33:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:33:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:33:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:33:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 00:33:51.645486 21671 mini_ranger.cc:160] Time spent starting Ranger: real 8.629s user 0.000s sys 0.010s
Aug 09, 2026 12:34:10 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:34:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 12:34:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:34:10 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:34:11 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 00:34:12.353067 21671 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.41.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.41.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/rangerkms_127.21.41.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/rangerkms_127.21.41.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.41.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.41.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.41.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/HTTP_127.21.41.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.41.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/HTTP_127.21.41.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 00:34:12 dist-test-slave-zpfg krb5kdc[32455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235652, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 00:34:12.440738 21671 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 00:34:12.451081 21671 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 00:34:13.413645 21671 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 00:34:13.413731 21671 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-kms
I20260809 00:34:13.413749 21671 mini_ranger_kms.cc:227] Using postgres at 127.21.41.212:57935
2026-08-09 00:34:14,113 [I] DB FLAVOR :POSTGRES
2026-08-09 00:34:14,113 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:34:14,113 [I] Checking connection
2026-08-09 00:34:14,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 00:34:14,427 [I] connection success
2026-08-09 00:34:14,427 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 00:34:14,427 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 00:34:14,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 00:34:14,745 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 00:34:14,745 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:34:14,745 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 00:34:14,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:57935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 00:34:15,046 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 00:34:15.486151 21671 mini_ranger_kms.cc:310] Created kms service
I20260809 00:34:15.681020 21671 mini_ranger_kms.cc:326] Created kudu user
I20260809 00:34:15.725656 21671 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 00:34:15 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235652, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:34:16.145969 21671 mini_ranger_kms.cc:384] Added ranger policy
I20260809 00:34:16.146108 21671 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-kms:/tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 00:34:16.146139 21671 mini_ranger_kms.cc:240] Using host: 127.21.41.212
I20260809 00:34:16.150246 21671 mini_ranger_kms.cc:276] Ranger KMS PID: 383
I20260809 00:34:16.150393 21671 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.41.212:60909
Aug 09, 2026 12:34:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:34:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:34:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:34:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:34:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:34:18 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 00:34:18.588661 21671 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.148s user 0.038s sys 0.271s
I20260809 00:34:18.588770 21671 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 00:34:18.588850 21671 mini_ranger_kms.cc:401] 127.21.41.212:60909/kms/v1/keys
Aug 09 00:34:19 dist-test-slave-zpfg krb5kdc[32455](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786235659, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.41.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 00:34:19 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235659, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.41.212@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
Aug 09 00:34:19 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235659, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.41.212@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
Aug 09 00:34:22 dist-test-slave-zpfg krb5kdc[32455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:34:22.290020 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:40875
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:40875
--ranger_config_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:60909/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:34:22.418944 435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:34:22.419198 435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:34:22.419224 435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:34:22.420527 435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:34:22.420574 435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:34:22.420590 435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:34:22.420610 435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:34:22.420655 435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:34:22.422595 435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.41.212:60909/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:40875
--ranger_config_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:40875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:34:22.422868 435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:34:22.423133 435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:34:22.428995 443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:34:22.429003 440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:34:22.429003 435 server_base.cc:1034] running on GCE node
W20260809 00:34:22.428995 441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:34:22.429560 435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:34:22.429934 435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:34:22.431137 435 hybrid_clock.cc:648] HybridClock initialized: now 1786235662431128 us; error 38 us; skew 500 ppm
Aug 09 00:34:22 dist-test-slave-zpfg krb5kdc[32455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:34:22.433833 435 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:34:22.434239 435 webserver.cc:460] Webserver started at http://127.21.41.254:39603/ using document root <none> and password file <none>
I20260809 00:34:22.434482 435 fs_manager.cc:359] Metadata directory not provided
I20260809 00:34:22.434525 435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:34:22.434635 435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:34:22 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:34:22.483204 435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "abd0d77773c44375b1f085784166bb94"
format_stamp: "Formatted at 2026-08-09 00:34:22 on dist-test-slave-zpfg"
server_key: "a34d9d52191be1c14232e7d5a4097ce9"
server_key_iv: "b5037614d4e8b58b83eb16582d8f2db1"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:22.483652 435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "abd0d77773c44375b1f085784166bb94"
format_stamp: "Formatted at 2026-08-09 00:34:22 on dist-test-slave-zpfg"
server_key: "a34d9d52191be1c14232e7d5a4097ce9"
server_key_iv: "b5037614d4e8b58b83eb16582d8f2db1"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:22.485553 435 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:34:22.497835 450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:34:22.498092 435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:34:22.498189 435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "abd0d77773c44375b1f085784166bb94"
format_stamp: "Formatted at 2026-08-09 00:34:22 on dist-test-slave-zpfg"
server_key: "a34d9d52191be1c14232e7d5a4097ce9"
server_key_iv: "b5037614d4e8b58b83eb16582d8f2db1"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:22.498322 435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:34:22.507658 435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:34:22.509644 435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:34:22.509817 435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:34:22.516613 435 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:40875
I20260809 00:34:22.516691 502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:40875 every 8 connection(s)
I20260809 00:34:22.517153 435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:34:22.518056 503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:34:22.521118 503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94: Bootstrap starting.
I20260809 00:34:22.522066 503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94: Neither blocks nor log segments found. Creating new log.
I20260809 00:34:22.522770 503 log.cc:824] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94: Log is configured to *not* fsync() on all Append() calls
I20260809 00:34:22.523589 503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94: No bootstrap required, opened a new log
I20260809 00:34:22.526909 503 raft_consensus.cc:348] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd0d77773c44375b1f085784166bb94" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40875 } }
I20260809 00:34:22.527091 503 raft_consensus.cc:374] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:34:22.527117 503 raft_consensus.cc:729] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd0d77773c44375b1f085784166bb94, State: Initialized, Role: FOLLOWER
I20260809 00:34:22.527310 503 consensus_queue.cc:260] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd0d77773c44375b1f085784166bb94" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40875 } }
I20260809 00:34:22.527375 503 raft_consensus.cc:388] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:34:22.527419 503 raft_consensus.cc:482] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:34:22.527417 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 435
I20260809 00:34:22.527468 503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:34:22.527514 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:34:22.527722 21671 external_mini_cluster.cc:1428] Setting key 8967b7783331cbeb6818cdff8e2356c3
I20260809 00:34:22.528353 503 raft_consensus.cc:504] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd0d77773c44375b1f085784166bb94" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40875 } }
I20260809 00:34:22.528470 503 leader_election.cc:302] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abd0d77773c44375b1f085784166bb94; no voters:
I20260809 00:34:22.528645 503 leader_election.cc:288] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:34:22.528750 508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:34:22.528895 503 sys_catalog.cc:564] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:34:22.528940 508 raft_consensus.cc:686] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [term 1 LEADER]: Becoming Leader. State: Replica: abd0d77773c44375b1f085784166bb94, State: Running, Role: LEADER
I20260809 00:34:22.529110 508 consensus_queue.cc:237] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd0d77773c44375b1f085784166bb94" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40875 } }
I20260809 00:34:22.529587 509 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abd0d77773c44375b1f085784166bb94. Latest consensus state: current_term: 1 leader_uuid: "abd0d77773c44375b1f085784166bb94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd0d77773c44375b1f085784166bb94" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40875 } } }
I20260809 00:34:22.529723 509 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [sys.catalog]: This master's current role is: LEADER
I20260809 00:34:22.529587 510 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abd0d77773c44375b1f085784166bb94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd0d77773c44375b1f085784166bb94" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40875 } } }
I20260809 00:34:22.529860 510 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94 [sys.catalog]: This master's current role is: LEADER
I20260809 00:34:22.530553 516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:34:22.531630 516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:34:22.531980 503 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 09 00:34:22 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:34:22.536203 525 mvcc.cc:205] Tried to move back new op lower bound from 7316421273734959104 to 7316421273719619584. Current Snapshot: MvccSnapshot[applied={T|T < 7316421273734959104}]
I20260809 00:34:22.543138 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:22.528362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36374 (local address 127.21.41.254:40875)
0809 00:34:22.528642 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 00:34:22.528653 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:34:22.528694 (+ 41us) server_negotiation.cc:407] Connection header received
0809 00:34:22.528828 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:34:22.528835 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:34:22.528891 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:34:22.530004 (+ 1113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:34:22.530094 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:22.531693 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:22.532899 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:22.533296 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:22.533369 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:22.537178 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:34:22.537213 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:34:22.537213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:34:22.537311 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:34:22.540368 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:34:22.540754 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:34:22.540757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:34:22.540757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:34:22.540936 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:34:22.541177 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:34:22.541178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:34:22.541179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:34:22.541650 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:34:22.542017 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0809 00:34:22.542097 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 00:34:22.542697 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":169,"threads_started":1}
I20260809 00:34:22.543412 516 catalog_manager.cc:1347] Generated new cluster ID: 91c9d0d5fed048cd97adfa4a330e4832
I20260809 00:34:22.543681 516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:34:22.565380 516 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:34:22.566440 516 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:34:22.574703 516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94: Generated new TSK 0
I20260809 00:34:22.574962 516 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:34:23.389889 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:60909/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40875
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:34:23.520435 548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:34:23.520656 548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:34:23.520699 548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:34:23.521971 548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:34:23.522011 548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:34:23.522038 548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:34:23.523515 548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.41.212:60909/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:34:23.523736 548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:34:23.523983 548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:34:23.529765 553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:34:23.530076 556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:34:23.530189 548 server_base.cc:1034] running on GCE node
W20260809 00:34:23.529778 554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:34:23.530512 548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:34:23.530777 548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:34:23.531939 548 hybrid_clock.cc:648] HybridClock initialized: now 1786235663531926 us; error 36 us; skew 500 ppm
Aug 09 00:34:23 dist-test-slave-zpfg krb5kdc[32455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:34:23.534916 548 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:34:23.535524 548 webserver.cc:460] Webserver started at http://127.21.41.193:34361/ using document root <none> and password file <none>
I20260809 00:34:23.535825 548 fs_manager.cc:359] Metadata directory not provided
I20260809 00:34:23.535902 548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:34:23.536062 548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:34:23 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:34:23.552875 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "350d380a6f434a3089b4047c02237340"
format_stamp: "Formatted at 2026-08-09 00:34:23 on dist-test-slave-zpfg"
server_key: "6832ce5f3af23d82f93927e8721dd513"
server_key_iv: "0d2bce3daa5ee66d8d5e7acc2e37bab2"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:23.553443 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "350d380a6f434a3089b4047c02237340"
format_stamp: "Formatted at 2026-08-09 00:34:23 on dist-test-slave-zpfg"
server_key: "6832ce5f3af23d82f93927e8721dd513"
server_key_iv: "0d2bce3daa5ee66d8d5e7acc2e37bab2"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:23.555619 548 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 00:34:23.571269 563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:34:23.571549 548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:34:23.571679 548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "350d380a6f434a3089b4047c02237340"
format_stamp: "Formatted at 2026-08-09 00:34:23 on dist-test-slave-zpfg"
server_key: "6832ce5f3af23d82f93927e8721dd513"
server_key_iv: "0d2bce3daa5ee66d8d5e7acc2e37bab2"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:23.571828 548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:34:23.601927 548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:34:23.604377 548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:34:23.604561 548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:34:23.604909 548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:34:23.605341 548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:34:23.605400 548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:34:23.605437 548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:34:23.605469 548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:34:23.614997 548 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:42085
I20260809 00:34:23.615067 679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:42085 every 8 connection(s)
I20260809 00:34:23.615550 548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:34:23.620137 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 548
I20260809 00:34:23.620258 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:34:23.620473 21671 external_mini_cluster.cc:1428] Setting key 4218e47510d817a8d3130dc25837ff39
Aug 09 00:34:23 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:34:23.627828 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:23.616318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:40875 (local address 127.21.41.193:57607)
0809 00:34:23.616572 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 00:34:23.616615 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:34:23.617134 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:34:23.617428 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:34:23.617431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:34:23.617612 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:34:23.617854 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:23.617860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:23.618897 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:23.618898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:23.620235 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:23.620238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:23.620805 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:23.620806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:23.620953 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:23.621730 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:34:23.621741 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:34:23.623626 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:34:23.625785 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:34:23.625792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:34:23.625795 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:34:23.626260 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:34:23.626535 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:34:23.626536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:34:23.626536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:34:23.626676 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:34:23.626993 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:34:23.626999 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:34:23.627185 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 00:34:23.627320 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 00:34:23.627507 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":85,"threads_started":1}
I20260809 00:34:23.628289 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:23.616392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:57607 (local address 127.21.41.254:40875)
0809 00:34:23.616507 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 00:34:23.616510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:34:23.616724 (+ 214us) server_negotiation.cc:407] Connection header received
0809 00:34:23.617245 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:34:23.617246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:34:23.617261 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:34:23.617354 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:34:23.617964 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:23.618777 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:23.620327 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:23.620631 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:23.620706 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:23.623744 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:34:23.623746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:34:23.623746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:34:23.623756 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:34:23.625686 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:34:23.626390 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:34:23.626392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:34:23.626392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:34:23.626459 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:34:23.626752 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:34:23.626753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:34:23.626754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:34:23.626901 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:34:23.626965 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:34:23.628105 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0809 00:34:23.628188 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260809 00:34:23.629273 680 heartbeater.cc:347] Connected to a master server at 127.21.41.254:40875
I20260809 00:34:23.629369 680 heartbeater.cc:464] Registering TS with master...
I20260809 00:34:23.630049 680 heartbeater.cc:511] Master 127.21.41.254:40875 requested a full tablet report, sending...
I20260809 00:34:23.630628 467 ts_manager.cc:194] Registered new tserver with Master: 350d380a6f434a3089b4047c02237340 (127.21.41.193:42085)
I20260809 00:34:23.631306 467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:57607
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:34:23.654218 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:60909/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40875
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:34:23 dist-test-slave-zpfg krb5kdc[32455](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786235663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 00:34:23.776538 686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:34:23.776935 686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:34:23.777019 686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:34:23.778836 686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:34:23.778939 686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:34:23.779047 686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:34:23.781385 686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.41.212:60909/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:34:23.781766 686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:34:23.782100 686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:34:23.788940 694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:34:23.789299 693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:34:23.789516 696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:34:23.790544 686 server_base.cc:1034] running on GCE node
I20260809 00:34:23.790745 686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:34:23.790997 686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:34:23.792171 686 hybrid_clock.cc:648] HybridClock initialized: now 1786235663792148 us; error 34 us; skew 500 ppm
Aug 09 00:34:23 dist-test-slave-zpfg krb5kdc[32455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:34:23.795612 686 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:34:23.796144 686 webserver.cc:460] Webserver started at http://127.21.41.194:46091/ using document root <none> and password file <none>
I20260809 00:34:23.796396 686 fs_manager.cc:359] Metadata directory not provided
I20260809 00:34:23.796456 686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:34:23.796626 686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:34:23 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:34:23.818631 686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "422041e1f144466b9b355cfabbe193af"
format_stamp: "Formatted at 2026-08-09 00:34:23 on dist-test-slave-zpfg"
server_key: "3192418fbeb3db804feafa7fa0fc7864"
server_key_iv: "aef77d85777e2fd7a2fecfce49d9d237"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:23.819137 686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "422041e1f144466b9b355cfabbe193af"
format_stamp: "Formatted at 2026-08-09 00:34:23 on dist-test-slave-zpfg"
server_key: "3192418fbeb3db804feafa7fa0fc7864"
server_key_iv: "aef77d85777e2fd7a2fecfce49d9d237"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:23.821166 686 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 00:34:23.848074 704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:34:23.849100 686 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260809 00:34:23.849258 686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "422041e1f144466b9b355cfabbe193af"
format_stamp: "Formatted at 2026-08-09 00:34:23 on dist-test-slave-zpfg"
server_key: "3192418fbeb3db804feafa7fa0fc7864"
server_key_iv: "aef77d85777e2fd7a2fecfce49d9d237"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:23.849383 686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:34:23.866408 686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:34:23.869019 686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:34:23.869242 686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:34:23.869515 686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:34:23.869943 686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:34:23.869994 686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:34:23.870028 686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:34:23.870045 686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:34:23.878506 686 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:37725
I20260809 00:34:23.879060 686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:34:23.879787 818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:37725 every 8 connection(s)
I20260809 00:34:23.882862 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 686
I20260809 00:34:23.882964 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:34:23.883180 21671 external_mini_cluster.cc:1428] Setting key 1bb86ba59499f1aa65c0d0558ad6524e
Aug 09 00:34:23 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:34:23.892371 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:23.879898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:37893 (local address 127.21.41.254:40875)
0809 00:34:23.879954 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:34:23.879956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:34:23.880195 (+ 239us) server_negotiation.cc:407] Connection header received
0809 00:34:23.880914 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:34:23.880916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:34:23.880934 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:34:23.881017 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:34:23.881832 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:23.882657 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:23.884149 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:23.884442 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:23.884493 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:23.887248 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:34:23.887250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:34:23.887250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:34:23.887258 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:34:23.889011 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:34:23.891238 (+ 2227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:34:23.891239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:34:23.891240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:34:23.891304 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:34:23.891597 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:34:23.891598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:34:23.891598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:34:23.891754 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:34:23.891796 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 00:34:23.892170 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 00:34:23.892266 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:34:23.893174 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:23.879878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:40875 (local address 127.21.41.194:37893)
0809 00:34:23.880031 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:34:23.880074 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:34:23.880769 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:34:23.881095 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:34:23.881100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:34:23.881380 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:34:23.881698 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:23.881705 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:23.882773 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:23.882774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:23.884038 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:23.884040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:23.884589 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:23.884590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:23.884747 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:23.885382 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:34:23.885393 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:34:23.887131 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:34:23.889113 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:34:23.889121 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:34:23.889123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:34:23.889420 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:34:23.891380 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:34:23.891380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:34:23.891381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:34:23.891521 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:34:23.891843 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:34:23.891849 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:34:23.892015 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 00:34:23.892744 (+ 729us) client_negotiation.cc:240] Negotiation successful
0809 00:34:23.892920 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260809 00:34:23.893837 819 heartbeater.cc:347] Connected to a master server at 127.21.41.254:40875
I20260809 00:34:23.894075 819 heartbeater.cc:464] Registering TS with master...
I20260809 00:34:23.894295 819 heartbeater.cc:511] Master 127.21.41.254:40875 requested a full tablet report, sending...
I20260809 00:34:23.894752 467 ts_manager.cc:194] Registered new tserver with Master: 422041e1f144466b9b355cfabbe193af (127.21.41.194:37725)
I20260809 00:34:23.895146 467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:37893
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:34:23.926496 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:60909/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40875
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:34:24.055027 824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:34:24.055336 824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:34:24.055382 824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:34:24.056978 824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:34:24.057057 824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:34:24.057103 824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:34:24.058750 824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.41.212:60909/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:40875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:34:24.059027 824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:34:24.059271 824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:34:24.065263 829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:34:24.065253 830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:34:24.065369 832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:34:24.065826 824 server_base.cc:1034] running on GCE node
I20260809 00:34:24.066025 824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:34:24.066267 824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:34:24.067435 824 hybrid_clock.cc:648] HybridClock initialized: now 1786235664067411 us; error 39 us; skew 500 ppm
Aug 09 00:34:24 dist-test-slave-zpfg krb5kdc[32455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:34:24.070034 824 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:34:24.070616 824 webserver.cc:460] Webserver started at http://127.21.41.195:35835/ using document root <none> and password file <none>
I20260809 00:34:24.070878 824 fs_manager.cc:359] Metadata directory not provided
I20260809 00:34:24.070922 824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:34:24.071062 824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:34:24 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:34:24.088084 824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "cf2d23eee3e5498e8827b15aec835ea5"
format_stamp: "Formatted at 2026-08-09 00:34:24 on dist-test-slave-zpfg"
server_key: "46189cc6497b174ade125b0e0b97ebc0"
server_key_iv: "7b221cdcfc9ed3a24717f89306b0ec44"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:24.088663 824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "cf2d23eee3e5498e8827b15aec835ea5"
format_stamp: "Formatted at 2026-08-09 00:34:24 on dist-test-slave-zpfg"
server_key: "46189cc6497b174ade125b0e0b97ebc0"
server_key_iv: "7b221cdcfc9ed3a24717f89306b0ec44"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:24.100433 824 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.002s sys 0.000s
I20260809 00:34:24.117177 839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:34:24.117376 824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:34:24.117481 824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "cf2d23eee3e5498e8827b15aec835ea5"
format_stamp: "Formatted at 2026-08-09 00:34:24 on dist-test-slave-zpfg"
server_key: "46189cc6497b174ade125b0e0b97ebc0"
server_key_iv: "7b221cdcfc9ed3a24717f89306b0ec44"
server_key_version: "kuduclusterkey@0"
I20260809 00:34:24.117595 824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:34:24.153954 824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:34:24.156538 824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:34:24.156736 824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:34:24.157001 824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:34:24.157441 824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:34:24.157498 824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:34:24.157531 824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:34:24.157593 824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:34:24.171777 824 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:41531
I20260809 00:34:24.172333 824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:34:24.175824 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 824
I20260809 00:34:24.175933 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:34:24.176158 21671 external_mini_cluster.cc:1428] Setting key 6c32b6ec63513d60f438712421bdc1ea
I20260809 00:34:24.176646 954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:41531 every 8 connection(s)
Aug 09 00:34:24 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:34:24.193048 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.175749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:46975 (local address 127.21.41.254:40875)
0809 00:34:24.175798 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:34:24.175800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:34:24.182670 (+ 6870us) server_negotiation.cc:407] Connection header received
0809 00:34:24.183154 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:34:24.183155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:34:24.183176 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:34:24.183256 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:34:24.184016 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.184769 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.186230 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.186536 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.186624 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.189927 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:34:24.189929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:34:24.189929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:34:24.189936 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:34:24.191575 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:34:24.192046 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:34:24.192047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:34:24.192048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:34:24.192102 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:34:24.192347 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:34:24.192347 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:34:24.192348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:34:24.192472 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:34:24.192509 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 00:34:24.192872 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 00:34:24.192956 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:34:24.193761 957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.175243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:40875 (local address 127.21.41.195:46975)
0809 00:34:24.182483 (+ 7240us) negotiation.cc:107] Waiting for socket to connect
0809 00:34:24.182541 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 00:34:24.183038 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:34:24.183359 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:34:24.183363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:34:24.183608 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:34:24.183903 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.183910 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.184886 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.184887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.186100 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.186103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.186639 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.186640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.186785 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.187601 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:34:24.187612 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:34:24.189821 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:34:24.191672 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:34:24.191679 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:34:24.191681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:34:24.191966 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:34:24.192164 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:34:24.192164 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:34:24.192165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:34:24.192280 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:34:24.192548 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:34:24.192553 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:34:24.192716 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 00:34:24.193376 (+ 660us) client_negotiation.cc:240] Negotiation successful
0809 00:34:24.193538 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7183,"thread_start_us":85,"threads_started":1}
I20260809 00:34:24.194221 955 heartbeater.cc:347] Connected to a master server at 127.21.41.254:40875
I20260809 00:34:24.194378 955 heartbeater.cc:464] Registering TS with master...
I20260809 00:34:24.194597 955 heartbeater.cc:511] Master 127.21.41.254:40875 requested a full tablet report, sending...
I20260809 00:34:24.195001 467 ts_manager.cc:194] Registered new tserver with Master: cf2d23eee3e5498e8827b15aec835ea5 (127.21.41.195:41531)
I20260809 00:34:24.195511 467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:46975
I20260809 00:34:24.199429 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:34:24.210073 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.201050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36384 (local address 127.21.41.254:40875)
0809 00:34:24.201106 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:34:24.201108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:34:24.201387 (+ 279us) server_negotiation.cc:407] Connection header received
0809 00:34:24.201502 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:34:24.201504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:34:24.201515 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:34:24.201544 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:34:24.202504 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.203259 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.204363 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.204646 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.204689 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.206362 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:34:24.206363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:34:24.206363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:34:24.206372 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:34:24.208663 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:34:24.209106 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:34:24.209108 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:34:24.209108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:34:24.209218 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:34:24.209459 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:34:24.209459 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:34:24.209460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:34:24.209614 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:34:24.209725 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 00:34:24.209879 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 00:34:24.209968 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:34:24.212087 467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:34:24.215023 467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:34:24.223470 967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.218333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:42085 (local address 127.0.0.1:59000)
0809 00:34:24.219158 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0809 00:34:24.219167 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:34:24.219210 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:34:24.219438 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:34:24.219441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:34:24.219444 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:34:24.219514 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.219516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.220815 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.220816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.222086 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.222089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.223119 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.223120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.223195 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.223211 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 00:34:24.223255 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 00:34:24.223359 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"thread_start_us":104,"threads_started":1}
I20260809 00:34:24.224011 972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.218611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59000 (local address 127.21.41.193:42085)
0809 00:34:24.218856 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 00:34:24.218862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:34:24.219256 (+ 394us) server_negotiation.cc:407] Connection header received
0809 00:34:24.219299 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:34:24.219301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:34:24.219318 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:34:24.219593 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:34:24.219650 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.220715 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.222183 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.222959 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.223805 (+ 846us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.223874 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:34:24.223901 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 00:34:24.223940 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":56,"threads_started":1}
I20260809 00:34:24.224089 971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.217870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:41531 (local address 127.0.0.1:38770)
0809 00:34:24.218507 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0809 00:34:24.218548 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:34:24.218629 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:34:24.219547 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:34:24.219547 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:34:24.219550 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:34:24.219612 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.219613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.221165 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.221166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.222359 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.222362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.223858 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.223859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.223923 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.223927 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:34:24.223975 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 00:34:24.224013 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":90,"threads_started":1}
I20260809 00:34:24.224670 968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.218010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38770 (local address 127.21.41.195:41531)
0809 00:34:24.219324 (+ 1314us) server_negotiation.cc:206] Beginning negotiation
0809 00:34:24.219329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:34:24.219339 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:34:24.219377 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:34:24.219379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:34:24.219395 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:34:24.219467 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:34:24.219696 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.221074 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.222478 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.223559 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.223641 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.223724 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:34:24.224540 (+ 816us) server_negotiation.cc:299] Negotiation successful
0809 00:34:24.224582 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1275,"thread_start_us":103,"threads_started":1}
I20260809 00:34:24.224951 614 tablet_service.cc:1467] Processing CreateTablet for tablet f7a15d46056e4ad6850c33d40d31b346 (DEFAULT_TABLE table=test-table [id=4631d3dee01c4725ac2827701c06c3c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:34:24.225327 614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7a15d46056e4ad6850c33d40d31b346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:34:24.225507 871 tablet_service.cc:1467] Processing CreateTablet for tablet f7a15d46056e4ad6850c33d40d31b346 (DEFAULT_TABLE table=test-table [id=4631d3dee01c4725ac2827701c06c3c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:34:24.225826 871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7a15d46056e4ad6850c33d40d31b346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:34:24.229038 974 tablet_bootstrap.cc:492] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5: Bootstrap starting.
I20260809 00:34:24.230260 974 tablet_bootstrap.cc:654] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5: Neither blocks nor log segments found. Creating new log.
I20260809 00:34:24.230675 974 log.cc:824] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:34:24.231567 974 tablet_bootstrap.cc:492] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5: No bootstrap required, opened a new log
I20260809 00:34:24.231633 974 ts_tablet_manager.cc:1397] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:34:24.232380 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.218138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:37725 (local address 127.0.0.1:40538)
0809 00:34:24.218988 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0809 00:34:24.218998 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:34:24.219077 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:34:24.226813 (+ 7736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:34:24.226814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:34:24.226817 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:34:24.226855 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.226856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.228128 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.228129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.229235 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.229238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.232123 (+ 2885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.232126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.232206 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.232211 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:34:24.232255 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 00:34:24.232295 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":72,"threads_started":1}
I20260809 00:34:24.232801 970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.218232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40538 (local address 127.21.41.194:37725)
0809 00:34:24.226609 (+ 8377us) server_negotiation.cc:206] Beginning negotiation
0809 00:34:24.226625 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 00:34:24.226638 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:34:24.226680 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:34:24.226682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:34:24.226699 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:34:24.226930 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:34:24.227011 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.228024 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.229966 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.232012 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.232553 (+ 541us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.232653 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 00:34:24.232679 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 00:34:24.232721 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8333,"thread_start_us":121,"threads_started":1}
I20260809 00:34:24.233413 734 tablet_service.cc:1467] Processing CreateTablet for tablet f7a15d46056e4ad6850c33d40d31b346 (DEFAULT_TABLE table=test-table [id=4631d3dee01c4725ac2827701c06c3c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:34:24.233505 974 raft_consensus.cc:348] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2d23eee3e5498e8827b15aec835ea5" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 41531 } } peers { permanent_uuid: "422041e1f144466b9b355cfabbe193af" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37725 } } peers { permanent_uuid: "350d380a6f434a3089b4047c02237340" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 42085 } }
I20260809 00:34:24.233665 974 raft_consensus.cc:374] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:34:24.233696 974 raft_consensus.cc:729] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf2d23eee3e5498e8827b15aec835ea5, State: Initialized, Role: FOLLOWER
I20260809 00:34:24.233788 734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7a15d46056e4ad6850c33d40d31b346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:34:24.233810 974 consensus_queue.cc:260] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2d23eee3e5498e8827b15aec835ea5" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 41531 } } peers { permanent_uuid: "422041e1f144466b9b355cfabbe193af" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37725 } } peers { permanent_uuid: "350d380a6f434a3089b4047c02237340" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 42085 } }
I20260809 00:34:24.234061 974 ts_tablet_manager.cc:1428] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:34:24.234634 955 heartbeater.cc:503] Master 127.21.41.254:40875 was elected leader, sending a full tablet report...
I20260809 00:34:24.237275 977 tablet_bootstrap.cc:492] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af: Bootstrap starting.
I20260809 00:34:24.237845 973 tablet_bootstrap.cc:492] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340: Bootstrap starting.
I20260809 00:34:24.238507 977 tablet_bootstrap.cc:654] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af: Neither blocks nor log segments found. Creating new log.
I20260809 00:34:24.238708 973 tablet_bootstrap.cc:654] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340: Neither blocks nor log segments found. Creating new log.
I20260809 00:34:24.238906 977 log.cc:824] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af: Log is configured to *not* fsync() on all Append() calls
I20260809 00:34:24.239094 973 log.cc:824] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340: Log is configured to *not* fsync() on all Append() calls
I20260809 00:34:24.239670 977 tablet_bootstrap.cc:492] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af: No bootstrap required, opened a new log
I20260809 00:34:24.239739 977 ts_tablet_manager.cc:1397] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 00:34:24.241642 977 raft_consensus.cc:348] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2d23eee3e5498e8827b15aec835ea5" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 41531 } } peers { permanent_uuid: "422041e1f144466b9b355cfabbe193af" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37725 } } peers { permanent_uuid: "350d380a6f434a3089b4047c02237340" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 42085 } }
I20260809 00:34:24.241815 977 raft_consensus.cc:374] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:34:24.241848 977 raft_consensus.cc:729] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 422041e1f144466b9b355cfabbe193af, State: Initialized, Role: FOLLOWER
I20260809 00:34:24.241971 977 consensus_queue.cc:260] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2d23eee3e5498e8827b15aec835ea5" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 41531 } } peers { permanent_uuid: "422041e1f144466b9b355cfabbe193af" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37725 } } peers { permanent_uuid: "350d380a6f434a3089b4047c02237340" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 42085 } }
I20260809 00:34:24.242244 977 ts_tablet_manager.cc:1428] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 00:34:24.242296 819 heartbeater.cc:503] Master 127.21.41.254:40875 was elected leader, sending a full tablet report...
I20260809 00:34:24.247496 973 tablet_bootstrap.cc:492] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340: No bootstrap required, opened a new log
I20260809 00:34:24.247695 973 ts_tablet_manager.cc:1397] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260809 00:34:24.250765 973 raft_consensus.cc:348] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2d23eee3e5498e8827b15aec835ea5" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 41531 } } peers { permanent_uuid: "422041e1f144466b9b355cfabbe193af" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37725 } } peers { permanent_uuid: "350d380a6f434a3089b4047c02237340" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 42085 } }
I20260809 00:34:24.251014 973 raft_consensus.cc:374] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:34:24.251098 973 raft_consensus.cc:729] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350d380a6f434a3089b4047c02237340, State: Initialized, Role: FOLLOWER
I20260809 00:34:24.251303 973 consensus_queue.cc:260] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2d23eee3e5498e8827b15aec835ea5" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 41531 } } peers { permanent_uuid: "422041e1f144466b9b355cfabbe193af" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37725 } } peers { permanent_uuid: "350d380a6f434a3089b4047c02237340" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 42085 } }
I20260809 00:34:24.251649 973 ts_tablet_manager.cc:1428] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 00:34:24.251817 680 heartbeater.cc:503] Master 127.21.41.254:40875 was elected leader, sending a full tablet report...
Aug 09 00:34:24 dist-test-slave-zpfg krb5kdc[32455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:34:24.277896 980 raft_consensus.cc:482] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:34:24.278050 980 raft_consensus.cc:504] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2d23eee3e5498e8827b15aec835ea5" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 41531 } } peers { permanent_uuid: "422041e1f144466b9b355cfabbe193af" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37725 } } peers { permanent_uuid: "350d380a6f434a3089b4047c02237340" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 42085 } }
I20260809 00:34:24.278522 980 leader_election.cc:288] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf2d23eee3e5498e8827b15aec835ea5 (127.21.41.195:41531), 350d380a6f434a3089b4047c02237340 (127.21.41.193:42085)
I20260809 00:34:24.285753 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.278674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:41531 (local address 127.21.41.194:56537)
0809 00:34:24.279087 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 00:34:24.279100 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:34:24.279146 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:34:24.282450 (+ 3304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:34:24.282452 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:34:24.282455 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:34:24.282534 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.282536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.283522 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.283523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.284745 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.284749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.285498 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.285499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.285580 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.285584 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:34:24.285633 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 00:34:24.285673 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395}
I20260809 00:34:24.286532 968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.278865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:56537 (local address 127.21.41.195:41531)
0809 00:34:24.279294 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 00:34:24.279296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:34:24.279305 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:34:24.279333 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:34:24.279334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:34:24.279345 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:34:24.279393 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:34:24.282639 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.283386 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.284854 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.285377 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.285452 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.285462 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:34:24.286427 (+ 965us) server_negotiation.cc:299] Negotiation successful
0809 00:34:24.286470 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403}
I20260809 00:34:24.286823 890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7a15d46056e4ad6850c33d40d31b346" candidate_uuid: "422041e1f144466b9b355cfabbe193af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf2d23eee3e5498e8827b15aec835ea5" is_pre_election: true
I20260809 00:34:24.286990 890 raft_consensus.cc:2454] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 422041e1f144466b9b355cfabbe193af in term 0.
I20260809 00:34:24.287252 708 leader_election.cc:302] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 422041e1f144466b9b355cfabbe193af, cf2d23eee3e5498e8827b15aec835ea5; no voters:
I20260809 00:34:24.287637 980 raft_consensus.cc:2781] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:34:24.287703 980 raft_consensus.cc:482] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:34:24.287729 980 raft_consensus.cc:3037] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:34:24.288738 980 raft_consensus.cc:504] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2d23eee3e5498e8827b15aec835ea5" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 41531 } } peers { permanent_uuid: "422041e1f144466b9b355cfabbe193af" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37725 } } peers { permanent_uuid: "350d380a6f434a3089b4047c02237340" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 42085 } }
I20260809 00:34:24.288878 980 leader_election.cc:288] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [CANDIDATE]: Term 1 election: Requested vote from peers cf2d23eee3e5498e8827b15aec835ea5 (127.21.41.195:41531), 350d380a6f434a3089b4047c02237340 (127.21.41.193:42085)
I20260809 00:34:24.289214 890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7a15d46056e4ad6850c33d40d31b346" candidate_uuid: "422041e1f144466b9b355cfabbe193af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf2d23eee3e5498e8827b15aec835ea5"
I20260809 00:34:24.289307 890 raft_consensus.cc:3037] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:34:24.290202 890 raft_consensus.cc:2454] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 422041e1f144466b9b355cfabbe193af in term 1.
I20260809 00:34:24.290439 708 leader_election.cc:302] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 422041e1f144466b9b355cfabbe193af, cf2d23eee3e5498e8827b15aec835ea5; no voters:
I20260809 00:34:24.291590 980 raft_consensus.cc:2781] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:34:24.291752 980 raft_consensus.cc:686] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 1 LEADER]: Becoming Leader. State: Replica: 422041e1f144466b9b355cfabbe193af, State: Running, Role: LEADER
I20260809 00:34:24.291859 980 consensus_queue.cc:237] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2d23eee3e5498e8827b15aec835ea5" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 41531 } } peers { permanent_uuid: "422041e1f144466b9b355cfabbe193af" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37725 } } peers { permanent_uuid: "350d380a6f434a3089b4047c02237340" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 42085 } }
I20260809 00:34:24.292872 466 catalog_manager.cc:5495] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af reported cstate change: term changed from 0 to 1, leader changed from <none> to 422041e1f144466b9b355cfabbe193af (127.21.41.194). New cstate: current_term: 1 leader_uuid: "422041e1f144466b9b355cfabbe193af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2d23eee3e5498e8827b15aec835ea5" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 41531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "422041e1f144466b9b355cfabbe193af" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 37725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "350d380a6f434a3089b4047c02237340" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 42085 } health_report { overall_health: UNKNOWN } } }
I20260809 00:34:24.297798 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.278913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:42085 (local address 127.21.41.194:59873)
0809 00:34:24.285834 (+ 6921us) negotiation.cc:107] Waiting for socket to connect
0809 00:34:24.285846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:34:24.285892 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:34:24.290590 (+ 4698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:34:24.290592 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:34:24.290594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:34:24.290633 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.290634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.293655 (+ 3021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.293657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.296630 (+ 2973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:34:24.296633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.297574 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.297575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:34:24.297636 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.297640 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:34:24.297678 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 00:34:24.297719 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6908,"thread_start_us":129,"threads_started":1}
I20260809 00:34:24.298151 972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.279544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:59873 (local address 127.21.41.193:42085)
0809 00:34:24.279589 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:34:24.279592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:34:24.286013 (+ 6421us) server_negotiation.cc:407] Connection header received
0809 00:34:24.286047 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:34:24.286048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:34:24.286063 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:34:24.286112 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:34:24.290731 (+ 4619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.291486 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.296744 (+ 5258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.297334 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.297409 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.297419 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:34:24.298034 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 00:34:24.298086 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:34:24.298729 633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7a15d46056e4ad6850c33d40d31b346" candidate_uuid: "422041e1f144466b9b355cfabbe193af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "350d380a6f434a3089b4047c02237340"
I20260809 00:34:24.298856 633 raft_consensus.cc:3037] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:34:24.299847 633 raft_consensus.cc:2454] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 422041e1f144466b9b355cfabbe193af in term 1.
I20260809 00:34:24.298707 634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7a15d46056e4ad6850c33d40d31b346" candidate_uuid: "422041e1f144466b9b355cfabbe193af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "350d380a6f434a3089b4047c02237340" is_pre_election: true
I20260809 00:34:24.300092 634 raft_consensus.cc:2362] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 422041e1f144466b9b355cfabbe193af in term 1. Re-sending same reply.
I20260809 00:34:24.304665 970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.301383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.21.41.194:37725)
0809 00:34:24.301431 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:34:24.301433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:34:24.301443 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:34:24.301540 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:34:24.301541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:34:24.301561 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:34:24.301625 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:34:24.301833 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.302508 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.303536 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:34:24.303808 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:34:24.303869 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:34:24.304088 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:34:24.304213 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:34:24.304309 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 00:34:24.304558 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 00:34:24.304580 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 00:34:24.306829 634 raft_consensus.cc:1261] T f7a15d46056e4ad6850c33d40d31b346 P 350d380a6f434a3089b4047c02237340 [term 1 FOLLOWER]: Refusing update from remote peer 422041e1f144466b9b355cfabbe193af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:34:24.306857 890 raft_consensus.cc:1261] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5 [term 1 FOLLOWER]: Refusing update from remote peer 422041e1f144466b9b355cfabbe193af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:34:24.307192 980 consensus_queue.cc:1035] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [LEADER]: Connected to new peer: Peer: permanent_uuid: "350d380a6f434a3089b4047c02237340" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 42085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:34:24.307353 980 consensus_queue.cc:1035] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2d23eee3e5498e8827b15aec835ea5" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 41531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:34:24.312481 987 mvcc.cc:205] Tried to move back new op lower bound from 7316421280997945344 to 7316421280940056576. Current Snapshot: MvccSnapshot[applied={T|T < 7316421280997945344}]
I20260809 00:34:24.314976 988 mvcc.cc:205] Tried to move back new op lower bound from 7316421280997945344 to 7316421280940056576. Current Snapshot: MvccSnapshot[applied={T|T < 7316421280997945344}]
I20260809 00:34:24.318241 467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:34:24.318384 467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:34:24.319161 467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abd0d77773c44375b1f085784166bb94: Sending DeleteTablet for 3 replicas of tablet f7a15d46056e4ad6850c33d40d31b346
I20260809 00:34:24.319698 871 tablet_service.cc:1514] Processing DeleteTablet for tablet f7a15d46056e4ad6850c33d40d31b346 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:34:24 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:38770
I20260809 00:34:24.319782 734 tablet_service.cc:1514] Processing DeleteTablet for tablet f7a15d46056e4ad6850c33d40d31b346 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:34:24 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:40538
I20260809 00:34:24.319919 614 tablet_service.cc:1514] Processing DeleteTablet for tablet f7a15d46056e4ad6850c33d40d31b346 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:34:24 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:59000
I20260809 00:34:24.320343 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 548
I20260809 00:34:24.330470 997 tablet_replica.cc:330] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5: stopping tablet replica
I20260809 00:34:24.330749 997 raft_consensus.cc:2229] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:34:24.331015 997 raft_consensus.cc:2258] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:34:24.331697 997 ts_tablet_manager.cc:1905] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:34:24.337973 997 ts_tablet_manager.cc:1918] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:34:24.338164 997 log.cc:1196] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/f7a15d46056e4ad6850c33d40d31b346
I20260809 00:34:24.338611 997 ts_tablet_manager.cc:1939] T f7a15d46056e4ad6850c33d40d31b346 P cf2d23eee3e5498e8827b15aec835ea5: Deleting consensus metadata
I20260809 00:34:24.339190 454 catalog_manager.cc:4841] TS cf2d23eee3e5498e8827b15aec835ea5 (127.21.41.195:41531): tablet f7a15d46056e4ad6850c33d40d31b346 (table test-table [id=4631d3dee01c4725ac2827701c06c3c2]) successfully deleted
I20260809 00:34:24.339494 998 tablet_replica.cc:330] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af: stopping tablet replica
I20260809 00:34:24.339689 998 raft_consensus.cc:2229] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:34:24.339921 998 raft_consensus.cc:2258] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:34:24.340567 998 ts_tablet_manager.cc:1905] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:34:24.342756 453 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:42085 (error 108)
I20260809 00:34:24.349915 998 ts_tablet_manager.cc:1918] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:34:24.350149 998 log.cc:1196] T f7a15d46056e4ad6850c33d40d31b346 P 422041e1f144466b9b355cfabbe193af: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/f7a15d46056e4ad6850c33d40d31b346
I20260809 00:34:24.350485 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 686
I20260809 00:34:24.353327 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:34:24.353169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:42085 (local address 127.0.0.1:59012)
0809 00:34:24.353231 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 00:34:24.353274 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:42085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 00:34:24.355180 453 catalog_manager.cc:4584] TS 350d380a6f434a3089b4047c02237340 (127.21.41.193:42085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7a15d46056e4ad6850c33d40d31b346: Network error: Client connection negotiation failed: client connection to 127.21.41.193:42085: connect: Connection refused (error 111)
W20260809 00:34:24.360337 969 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:34:24.359986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:37725 (local address 127.0.0.1:40554)
0809 00:34:24.360031 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 00:34:24.360042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:34:24.360082 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:34:24.360286 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.194:37725: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 00:34:24.360901 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 824
I20260809 00:34:24.367681 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 435
I20260809 00:34:24.373708 21671 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 00:34:24.718317 21671 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 00:34:24.718492 21671 mini_ranger.cc:66] Stopping Ranger...
I20260809 00:34:25.063557 21671 mini_ranger.cc:68] Stopped Ranger
2026-08-09 00:34:25.063 UTC [32485] LOG: received smart shutdown request
2026-08-09 00:34:25.067 UTC [32485] LOG: background worker "logical replication launcher" (PID 32494) exited with exit code 1
2026-08-09 00:34:25.067 UTC [32489] LOG: shutting down
2026-08-09 00:34:25.241 UTC [32485] LOG: database system is shut down
2026-08-09T00:34:25Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47322 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:34:25 dist-test-slave-zpfg krb5kdc[1009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:34:25 dist-test-slave-zpfg krb5kdc[1009](info): set up 2 sockets
Aug 09 00:34:25 dist-test-slave-zpfg krb5kdc[1009](info): commencing operation
krb5kdc: starting...
W20260809 00:34:26.392287 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:34:26 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:34:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:34:26Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.41.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.41.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/rangeradmin_127.21.41.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/rangeradmin_127.21.41.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.41.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.41.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/rangerlookup_127.21.41.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/rangerlookup_127.21.41.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.41.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.41.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/HTTP_127.21.41.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/HTTP_127.21.41.212@KRBTEST.COM.keytab.
I20260809 00:34:26.561373 21671 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskbJF9WP/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/postgres -l logfile start
2026-08-09 00:34:29.454 UTC [1039] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 00:34:29.454 UTC [1039] LOG: listening on IPv4 address "127.21.41.212", port 34561
2026-08-09 00:34:29.458 UTC [1039] LOG: listening on Unix socket "/tmp/.s.PGSQL.34561"
2026-08-09 00:34:29.479 UTC [1042] LOG: database system was shut down at 2026-08-09 00:34:27 UTC
2026-08-09 00:34:29.483 UTC [1039] LOG: database system is ready to accept connections
I20260809 00:34:30.503890 21671 mini_postgres.cc:96] Postgres bound to 34561
2026-08-09 00:34:30.510 UTC [1050] FATAL: database "slave" does not exist
127.21.41.212:34561 - accepting connections
I20260809 00:34:30.510946 21671 mini_ranger.cc:161] Starting Ranger...
I20260809 00:34:30.525701 21671 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 00:34:31.030429 21671 mini_ranger.cc:87] Created ranger Postgres database
I20260809 00:34:31.030507 21671 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-admin
2026-08-09 00:34:31,329 [I] DB FLAVOR :POSTGRES
2026-08-09 00:34:31,329 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:34:31,330 [I] Checking connection..
2026-08-09 00:34:31,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 00:34:31,561 [I] Checking connection passed.
2026-08-09 00:34:31,561 [I] --------- Verifying version history table ---------
2026-08-09 00:34:31,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:34:31,823 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:34:31,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:34:32,085 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:34:32,086 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 00:34:32,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 00:34:32.322 UTC [1115] WARNING: there is no transaction in progress
2026-08-09 00:34:32,329 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 00:34:32,329 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:34:32,597 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 00:34:32,597 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:34:32,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 00:34:32,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 00:34:33,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zpfg.c.gcp-upstream.internal','N') ;"
2026-08-09 00:34:33,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:34:33,563 [I] Table x_portal_user does not exist in database ranger
2026-08-09 00:34:33,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:34:33,811 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 00:34:33,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:34:34,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:34:34,325 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 00:34:34,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 00:34:35.315 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.331 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.349 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.365 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.386 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.407 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.472 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.479 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.494 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.504 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.514 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.527 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.536 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.543 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.549 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:35.556 UTC [1224] WARNING: there is no transaction in progress
2026-08-09 00:34:36,197 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 00:34:36,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 00:34:36,421 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 00:34:36,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:34:36,661 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 00:34:36,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:34:36,904 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 00:34:36,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:34:37,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:34:37,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zpfg.c.gcp-upstream.internal';"
2026-08-09 00:34:37,631 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 00:34:37,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 00:34:37,856 [I] DB_PATCHES have already been applied
I20260809 00:34:37.861646 21671 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-admin:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskbJF9WP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 00:34:37.861740 21671 mini_ranger.cc:193] Using host: 127.21.41.212
I20260809 00:34:37.865132 21671 mini_ranger.cc:227] Ranger admin URL: http://127.21.41.212:36237
Aug 09, 2026 12:34:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:34:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:34:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:34:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:34:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:34:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:34:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 00:34:38.984004 21671 mini_ranger.cc:160] Time spent starting Ranger: real 8.473s user 0.001s sys 0.009s
Aug 09, 2026 12:34:59 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:34:59 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 12:34:59 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:34:59 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:35:00 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 00:35:01.095427 21671 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.41.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.41.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/rangerkms_127.21.41.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.41.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/rangerkms_127.21.41.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.41.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.41.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.41.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/HTTP_127.21.41.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.41.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/HTTP_127.21.41.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 00:35:01 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235701, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 00:35:01.187085 21671 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 00:35:01.196403 21671 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 00:35:02.096791 21671 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 00:35:02.096881 21671 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-kms
I20260809 00:35:02.096904 21671 mini_ranger_kms.cc:227] Using postgres at 127.21.41.212:34561
2026-08-09 00:35:02,262 [I] DB FLAVOR :POSTGRES
2026-08-09 00:35:02,263 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:35:02,263 [I] Checking connection
2026-08-09 00:35:02,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 00:35:02,502 [I] connection success
2026-08-09 00:35:02,503 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 00:35:02,503 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 00:35:02,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 00:35:02,769 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 00:35:02,769 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:35:02,769 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 00:35:02,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.41.212:34561/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 00:35:03,032 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 00:35:03.512130 21671 mini_ranger_kms.cc:310] Created kms service
I20260809 00:35:03.697165 21671 mini_ranger_kms.cc:326] Created kudu user
I20260809 00:35:03.741735 21671 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 00:35:03 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235701, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:35:04.136766 21671 mini_ranger_kms.cc:384] Added ranger policy
I20260809 00:35:04.136922 21671 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-kms:/tmp/dist-test-taskbJF9WP/build/release/bin/postgresql.jar:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskbJF9WP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 00:35:04.136943 21671 mini_ranger_kms.cc:240] Using host: 127.21.41.212
I20260809 00:35:04.140866 21671 mini_ranger_kms.cc:276] Ranger KMS PID: 1405
I20260809 00:35:04.140952 21671 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.41.212:35027
Aug 09, 2026 12:35:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:35:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:35:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:35:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:35:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:35:05 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 00:35:06 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786235706, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.41.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:06.537917 21671 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.351s user 0.003s sys 0.146s
I20260809 00:35:06.538021 21671 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 00:35:06.538071 21671 mini_ranger_kms.cc:401] 127.21.41.212:35027/kms/v1/keys
Aug 09 00:35:06 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235706, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.41.212@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
Aug 09 00:35:07 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235706, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.41.212@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
Aug 09 00:35:09 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:35:09.545457 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42881
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:42881
--ranger_config_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:35027/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:09.662760 1456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:09.663014 1456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:09.663053 1456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:09.664247 1456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:35:09.664316 1456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:09.664342 1456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:09.664363 1456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:35:09.664423 1456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:35:09.666033 1456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.41.212:35027/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:42881
--ranger_config_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:09.666311 1456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:09.666567 1456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:09.671856 1461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:09.671895 1464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:09.671942 1462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:09.672151 1456 server_base.cc:1034] running on GCE node
I20260809 00:35:09.672411 1456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:09.672653 1456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:09.673782 1456 hybrid_clock.cc:648] HybridClock initialized: now 1786235709673757 us; error 43 us; skew 500 ppm
Aug 09 00:35:09 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235709, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:09.681277 1456 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:35:09.681811 1456 webserver.cc:460] Webserver started at http://127.21.41.254:43503/ using document root <none> and password file <none>
I20260809 00:35:09.682063 1456 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:09.682125 1456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:09.682279 1456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:35:09 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235709, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:35:09.727406 1456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "90be3d7a5bb64a039433024bfafa1c78"
format_stamp: "Formatted at 2026-08-09 00:35:09 on dist-test-slave-zpfg"
server_key: "182de9a30a5a0d67a34e39b3fe481279"
server_key_iv: "e46629dcc818222eaf09c0864e9bd2ab"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:09.727886 1456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "90be3d7a5bb64a039433024bfafa1c78"
format_stamp: "Formatted at 2026-08-09 00:35:09 on dist-test-slave-zpfg"
server_key: "182de9a30a5a0d67a34e39b3fe481279"
server_key_iv: "e46629dcc818222eaf09c0864e9bd2ab"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:09.729766 1456 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:35:09.741304 1471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:09.741492 1456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:35:09.741590 1456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "90be3d7a5bb64a039433024bfafa1c78"
format_stamp: "Formatted at 2026-08-09 00:35:09 on dist-test-slave-zpfg"
server_key: "182de9a30a5a0d67a34e39b3fe481279"
server_key_iv: "e46629dcc818222eaf09c0864e9bd2ab"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:09.741691 1456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:09.753101 1456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:09.755292 1456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:09.755466 1456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:09.760027 1456 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:42881
I20260809 00:35:09.760067 1524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:42881 every 8 connection(s)
I20260809 00:35:09.760417 1456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:35:09.761091 1525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:09.762483 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 1456
I20260809 00:35:09.762583 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:35:09.762794 21671 external_mini_cluster.cc:1428] Setting key 3207c3892070274d89641399d4623853
I20260809 00:35:09.763942 1525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78: Bootstrap starting.
I20260809 00:35:09.764676 1525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:09.764989 1525 log.cc:824] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:09.765786 1525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78: No bootstrap required, opened a new log
I20260809 00:35:09.767992 1525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } }
I20260809 00:35:09.768162 1525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:09.768196 1525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90be3d7a5bb64a039433024bfafa1c78, State: Initialized, Role: FOLLOWER
I20260809 00:35:09.768373 1525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } }
I20260809 00:35:09.768571 1525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:35:09.768652 1525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:35:09.768736 1525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:09.769384 1525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } }
I20260809 00:35:09.769511 1525 leader_election.cc:302] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90be3d7a5bb64a039433024bfafa1c78; no voters:
I20260809 00:35:09.769713 1525 leader_election.cc:288] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:35:09.769742 1530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:35:09.770056 1530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 1 LEADER]: Becoming Leader. State: Replica: 90be3d7a5bb64a039433024bfafa1c78, State: Running, Role: LEADER
I20260809 00:35:09.770071 1525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:35:09.770269 1530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } }
I20260809 00:35:09.770696 1530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90be3d7a5bb64a039433024bfafa1c78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } } }
Aug 09 00:35:09 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:09.770870 1530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:09.771005 1530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90be3d7a5bb64a039433024bfafa1c78. Latest consensus state: current_term: 1 leader_uuid: "90be3d7a5bb64a039433024bfafa1c78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } } }
I20260809 00:35:09.771123 1530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:09.771291 1539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:35:09.771740 1525 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 00:35:09.781844 1539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:35:09.783254 1528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:09.763359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37112 (local address 127.21.41.254:42881)
0809 00:35:09.763572 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:09.763583 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:09.763644 (+ 61us) server_negotiation.cc:407] Connection header received
0809 00:35:09.764199 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:09.764208 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:09.764277 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:09.764440 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:09.765220 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:09.766616 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:09.767695 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:09.768324 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:09.768474 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:09.771443 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:09.771447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:09.771447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:09.771460 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:09.775736 (+ 4276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:09.781156 (+ 5420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:09.781167 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:09.781168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:09.781323 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:09.781614 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:09.781615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:09.781616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:09.782020 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:09.782183 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:09.782804 (+ 621us) server_negotiation.cc:299] Negotiation successful
0809 00:35:09.783004 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":88,"threads_started":1}
I20260809 00:35:09.786866 1539 catalog_manager.cc:1347] Generated new cluster ID: d5844cc6083145b38b25c30c0dd6d4c5
I20260809 00:35:09.786926 1539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:35:09.813695 1539 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:35:09.814574 1539 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:35:09.822839 1539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78: Generated new TSK 0
I20260809 00:35:09.823112 1539 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:35:10.513681 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:35027/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42881
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:10.634991 1570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:10.635349 1570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:10.635424 1570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:10.637176 1570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:10.637295 1570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:10.637390 1570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:35:10.639578 1570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.41.212:35027/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:10.640060 1570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:10.640422 1570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:10.645826 1576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:10.645826 1575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:10.645844 1578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:10.646726 1570 server_base.cc:1034] running on GCE node
I20260809 00:35:10.646950 1570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:10.647241 1570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:10.648435 1570 hybrid_clock.cc:648] HybridClock initialized: now 1786235710648411 us; error 35 us; skew 500 ppm
Aug 09 00:35:10 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:10.651464 1570 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:35:10.651927 1570 webserver.cc:460] Webserver started at http://127.21.41.193:35645/ using document root <none> and password file <none>
I20260809 00:35:10.652164 1570 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:10.652258 1570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:10.652407 1570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:35:10 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:35:10.666704 1570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "06464e21356a4f2fbdbaa3c6197aad3a"
format_stamp: "Formatted at 2026-08-09 00:35:10 on dist-test-slave-zpfg"
server_key: "9e71f8d4484d44aefac4a64babaaf11b"
server_key_iv: "5f27537c3e019a65eb624f90e1dcc45c"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:10.667251 1570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "06464e21356a4f2fbdbaa3c6197aad3a"
format_stamp: "Formatted at 2026-08-09 00:35:10 on dist-test-slave-zpfg"
server_key: "9e71f8d4484d44aefac4a64babaaf11b"
server_key_iv: "5f27537c3e019a65eb624f90e1dcc45c"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:10.668746 1570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:10.683563 1587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:10.683883 1570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:10.684015 1570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "06464e21356a4f2fbdbaa3c6197aad3a"
format_stamp: "Formatted at 2026-08-09 00:35:10 on dist-test-slave-zpfg"
server_key: "9e71f8d4484d44aefac4a64babaaf11b"
server_key_iv: "5f27537c3e019a65eb624f90e1dcc45c"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:10.684134 1570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:10.699280 1570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:10.701829 1570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:10.701983 1570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:10.702239 1570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:10.702630 1570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:10.702678 1570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:10.702708 1570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:10.702726 1570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:10.709082 1570 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:41581
I20260809 00:35:10.709156 1702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:41581 every 8 connection(s)
I20260809 00:35:10.709483 1570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:35:10.711064 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 1570
I20260809 00:35:10.711169 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:35:10.711402 21671 external_mini_cluster.cc:1428] Setting key b45bd2fe62676e84d0ee8c618180db31
Aug 09 00:35:10 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
I20260809 00:35:10.721455 1705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:10.710199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42881 (local address 127.21.41.193:38263)
0809 00:35:10.710458 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:10.710510 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:10.711304 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:10.711694 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:10.711698 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:10.711969 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:10.712254 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:10.712261 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:10.713201 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:10.713203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:10.714502 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:10.714505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:10.715057 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:10.715058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:10.715207 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:10.715869 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:10.715879 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:10.717565 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:10.719756 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:10.719763 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:10.719766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:10.720066 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:10.720295 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:10.720296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:10.720296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:10.720429 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:10.720760 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:10.720767 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:10.720929 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 00:35:10.721042 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 00:35:10.721207 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260809 00:35:10.721812 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:10.710278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:38263 (local address 127.21.41.254:42881)
0809 00:35:10.710460 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:10.710462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:10.710611 (+ 149us) server_negotiation.cc:407] Connection header received
0809 00:35:10.711477 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:10.711478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:10.711493 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:10.711556 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:10.712361 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:10.713092 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:10.714613 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:10.714939 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:10.715017 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:10.717689 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:10.717691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:10.717691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:10.717699 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:10.719643 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:10.720158 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:10.720159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:10.720159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:10.720224 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:10.720505 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:10.720505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:10.720506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:10.720654 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:10.720719 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:10.721607 (+ 888us) server_negotiation.cc:299] Negotiation successful
0809 00:35:10.721701 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
I20260809 00:35:10.722365 1703 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42881
I20260809 00:35:10.722473 1703 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:10.722667 1703 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
I20260809 00:35:10.723299 1489 ts_manager.cc:194] Registered new tserver with Master: 06464e21356a4f2fbdbaa3c6197aad3a (127.21.41.193:41581)
I20260809 00:35:10.724207 1489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:38263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:35:10.741850 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:35027/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42881
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:35:10 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786235710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 00:35:10.885540 1709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:10.885787 1709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:10.885833 1709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:10.887666 1709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:10.887810 1709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:10.887952 1709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:35:10.889735 1709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.41.212:35027/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:10.890025 1709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:10.890319 1709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:10.900048 1716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:10.900360 1717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:10.902521 1719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:10.903833 1709 server_base.cc:1034] running on GCE node
I20260809 00:35:10.904160 1709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:10.904438 1709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:10.905613 1709 hybrid_clock.cc:648] HybridClock initialized: now 1786235710905592 us; error 35 us; skew 500 ppm
Aug 09 00:35:10 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:10.909736 1709 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:35:10.910291 1709 webserver.cc:460] Webserver started at http://127.21.41.194:46067/ using document root <none> and password file <none>
I20260809 00:35:10.910610 1709 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:10.910743 1709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:10.910938 1709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:35:10 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:35:10.924444 1709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "a8f3fcedf217413e867c09a788cfa765"
format_stamp: "Formatted at 2026-08-09 00:35:10 on dist-test-slave-zpfg"
server_key: "bde5545b437ba04cfa3376bbc8ffb957"
server_key_iv: "2c33d83e59989de934ce4cd559e5637b"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:10.924991 1709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "a8f3fcedf217413e867c09a788cfa765"
format_stamp: "Formatted at 2026-08-09 00:35:10 on dist-test-slave-zpfg"
server_key: "bde5545b437ba04cfa3376bbc8ffb957"
server_key_iv: "2c33d83e59989de934ce4cd559e5637b"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:10.931190 1709 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.000s
I20260809 00:35:10.949100 1727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:10.949456 1709 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260809 00:35:10.949592 1709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "a8f3fcedf217413e867c09a788cfa765"
format_stamp: "Formatted at 2026-08-09 00:35:10 on dist-test-slave-zpfg"
server_key: "bde5545b437ba04cfa3376bbc8ffb957"
server_key_iv: "2c33d83e59989de934ce4cd559e5637b"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:10.949733 1709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:11.012722 1709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:11.015342 1709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:11.015519 1709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:11.015838 1709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:11.016273 1709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:11.016324 1709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.016355 1709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:11.016373 1709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.024462 1709 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:45641
I20260809 00:35:11.024592 1843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:45641 every 8 connection(s)
I20260809 00:35:11.024964 1709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:35:11.029687 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 1709
I20260809 00:35:11.029805 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:35:11.030030 21671 external_mini_cluster.cc:1428] Setting key 97cf7e7169518a66d0195c91e2d5937d
Aug 09 00:35:11 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:11.035972 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:11.025881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:50371 (local address 127.21.41.254:42881)
0809 00:35:11.025941 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:11.025943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:11.026214 (+ 271us) server_negotiation.cc:407] Connection header received
0809 00:35:11.026851 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:11.026853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:11.026871 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:11.026955 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:11.027491 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.028364 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.029872 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.030080 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.030166 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:11.032524 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:11.032526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:11.032526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:11.032534 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:11.034515 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:11.034938 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:11.034939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:11.034940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:11.035032 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:11.035292 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:11.035293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:11.035293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:11.035415 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:11.035486 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:11.035772 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 00:35:11.035862 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:35:11.036151 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:11.025864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42881 (local address 127.21.41.194:50371)
0809 00:35:11.026041 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:11.026091 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:11.026724 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:11.027006 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:11.027010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:11.027201 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:11.027396 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:11.027400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.028478 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.028479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:11.029754 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:11.029757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.030184 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.030185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:11.030335 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:11.031003 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:11.031014 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:11.032399 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:11.034620 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:11.034626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:11.034628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:11.034847 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:11.035105 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:11.035105 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:11.035106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:11.035224 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:11.035488 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:11.035495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:11.035642 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 00:35:11.035751 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 00:35:11.035892 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260809 00:35:11.036677 1844 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42881
I20260809 00:35:11.036770 1844 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:11.036949 1844 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
I20260809 00:35:11.037343 1489 ts_manager.cc:194] Registered new tserver with Master: a8f3fcedf217413e867c09a788cfa765 (127.21.41.194:45641)
I20260809 00:35:11.037786 1489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:50371
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:35:11.062305 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:35027/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42881
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:11.160153 1849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:11.160386 1849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:11.160427 1849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:11.161594 1849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:11.161657 1849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:11.161710 1849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:35:11.163309 1849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.41.212:35027/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:11.163568 1849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:11.163810 1849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:11.169098 1855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:11.169117 1854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:11.169191 1857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:11.169590 1849 server_base.cc:1034] running on GCE node
I20260809 00:35:11.169842 1849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:11.170094 1849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:11.171283 1849 hybrid_clock.cc:648] HybridClock initialized: now 1786235711171245 us; error 56 us; skew 500 ppm
Aug 09 00:35:11 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:11.174013 1849 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:35:11.174595 1849 webserver.cc:460] Webserver started at http://127.21.41.195:42889/ using document root <none> and password file <none>
I20260809 00:35:11.174860 1849 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:11.174928 1849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:11.175071 1849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:35:11 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:35:11.191483 1849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "6056018aad3a43d08e9a7204fd08445b"
format_stamp: "Formatted at 2026-08-09 00:35:11 on dist-test-slave-zpfg"
server_key: "14e03fa47c041e6c95ad3c5e617eb1f6"
server_key_iv: "060fc7169014572433391073db356bb5"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:11.191983 1849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "6056018aad3a43d08e9a7204fd08445b"
format_stamp: "Formatted at 2026-08-09 00:35:11 on dist-test-slave-zpfg"
server_key: "14e03fa47c041e6c95ad3c5e617eb1f6"
server_key_iv: "060fc7169014572433391073db356bb5"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:11.202783 1849 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.002s sys 0.000s
I20260809 00:35:11.219987 1864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.220227 1849 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 00:35:11.220331 1849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "6056018aad3a43d08e9a7204fd08445b"
format_stamp: "Formatted at 2026-08-09 00:35:11 on dist-test-slave-zpfg"
server_key: "14e03fa47c041e6c95ad3c5e617eb1f6"
server_key_iv: "060fc7169014572433391073db356bb5"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:11.220443 1849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:11.250947 1849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:11.253935 1849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:11.254191 1849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:11.254607 1849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:11.255107 1849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:11.255213 1849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.255287 1849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:11.255347 1849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.262048 1849 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:43005
I20260809 00:35:11.264756 1978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:43005 every 8 connection(s)
I20260809 00:35:11.265231 1849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:35:11.270191 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 1849
I20260809 00:35:11.270314 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:35:11.270606 21671 external_mini_cluster.cc:1428] Setting key 3eca158e562e3446bf8716744b549bdc
Aug 09 00:35:11 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:11.278090 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:11.266460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:53993 (local address 127.21.41.254:42881)
0809 00:35:11.266507 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:11.266510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:11.266964 (+ 454us) server_negotiation.cc:407] Connection header received
0809 00:35:11.267701 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:11.267703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:11.267715 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:11.267747 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:11.268670 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.269518 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.271072 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.271392 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.271440 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:11.274244 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:11.274246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:11.274246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:11.274254 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:11.276374 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:11.276892 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:11.276893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:11.276894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:11.276965 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:11.277298 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:11.277299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:11.277299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:11.277455 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:11.277537 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:11.277889 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 00:35:11.277982 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:35:11.278235 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:11.266546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42881 (local address 127.21.41.195:53993)
0809 00:35:11.266815 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:11.266861 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:11.267568 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:11.267935 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:11.267939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:11.268209 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:11.268502 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:11.268509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.269633 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.269634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:11.270962 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:11.270965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.271579 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.271580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:11.271725 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:11.272359 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:11.272369 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:11.274118 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:11.276472 (+ 2354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:11.276480 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:11.276482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:11.276794 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:11.277052 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:11.277053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:11.277053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:11.277205 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:11.277538 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:11.277544 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:11.277706 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 00:35:11.277833 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 00:35:11.277991 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":86,"threads_started":1}
I20260809 00:35:11.278681 1979 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42881
I20260809 00:35:11.278779 1979 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:11.278966 1979 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
I20260809 00:35:11.279420 1489 ts_manager.cc:194] Registered new tserver with Master: 6056018aad3a43d08e9a7204fd08445b (127.21.41.195:43005)
I20260809 00:35:11.279927 1489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:53993
I20260809 00:35:11.282380 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:35:11.283179 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 1570
I20260809 00:35:11.291260 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 1709
I20260809 00:35:11.299015 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 1849
I20260809 00:35:11.305227 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 1456
I20260809 00:35:11.310621 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42881
--webserver_interface=127.21.41.254
--webserver_port=43503
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:42881
--ranger_config_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:35027/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.41.212:35027/kms with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:11.393043 1982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:11.393244 1982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:11.393266 1982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:11.394400 1982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:35:11.394438 1982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:11.394454 1982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:11.394466 1982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:35:11.394479 1982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:35:11.395977 1982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.41.212:35027/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:42881
--ranger_config_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=43503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:11.396162 1982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:11.396323 1982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:11.401497 1990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:11.401662 1982 server_base.cc:1034] running on GCE node
W20260809 00:35:11.401515 1987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:11.401556 1988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:11.401997 1982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:11.402242 1982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:11.403419 1982 hybrid_clock.cc:648] HybridClock initialized: now 1786235711403405 us; error 44 us; skew 500 ppm
Aug 09 00:35:11 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:11.406070 1982 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:35:11.406639 1982 webserver.cc:460] Webserver started at http://127.21.41.254:43503/ using document root <none> and password file <none>
I20260809 00:35:11.406831 1982 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:11.406881 1982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:35:11 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:35:11.484236 1982 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:11.485028 1998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.485203 1982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:11.485303 1982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "90be3d7a5bb64a039433024bfafa1c78"
format_stamp: "Formatted at 2026-08-09 00:35:09 on dist-test-slave-zpfg"
server_key: "182de9a30a5a0d67a34e39b3fe481279"
server_key_iv: "e46629dcc818222eaf09c0864e9bd2ab"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:11.485685 1982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:11.542125 1982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:11.544384 1982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:11.544557 1982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:11.548949 1982 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:42881
I20260809 00:35:11.549007 2050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:42881 every 8 connection(s)
I20260809 00:35:11.549397 1982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:35:11.550808 2051 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:35:11.551523 2051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78: Bootstrap starting.
I20260809 00:35:11.557876 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 1982
I20260809 00:35:11.558175 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:35027/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:41581
--local_ip_for_outbound_sockets=127.21.41.193
--tserver_master_addrs=127.21.41.254:42881
--webserver_port=35645
--webserver_interface=127.21.41.193
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.41.212:35027/kms with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:35:11.560243 2051 log.cc:824] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:11.562285 2051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:35:11.562639 2051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78: Bootstrap complete.
I20260809 00:35:11.565260 2051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } }
I20260809 00:35:11.565541 2051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90be3d7a5bb64a039433024bfafa1c78, State: Initialized, Role: FOLLOWER
I20260809 00:35:11.565788 2051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } }
I20260809 00:35:11.565892 2051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:35:11.565950 2051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:35:11.566004 2051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:35:11.567178 2051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } }
I20260809 00:35:11.567308 2051 leader_election.cc:302] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90be3d7a5bb64a039433024bfafa1c78; no voters:
I20260809 00:35:11.567526 2051 leader_election.cc:288] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:35:11.567564 2055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:35:11.567687 2055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [term 2 LEADER]: Becoming Leader. State: Replica: 90be3d7a5bb64a039433024bfafa1c78, State: Running, Role: LEADER
I20260809 00:35:11.567832 2051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:35:11.567843 2055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } }
I20260809 00:35:11.568282 2057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90be3d7a5bb64a039433024bfafa1c78. Latest consensus state: current_term: 2 leader_uuid: "90be3d7a5bb64a039433024bfafa1c78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } } }
I20260809 00:35:11.568372 2057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:11.568698 2061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:35:11.568737 2056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "90be3d7a5bb64a039433024bfafa1c78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90be3d7a5bb64a039433024bfafa1c78" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42881 } } }
I20260809 00:35:11.568866 2056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:11.569371 2061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:35:11.569540 2061 catalog_manager.cc:1259] Loaded cluster ID: d5844cc6083145b38b25c30c0dd6d4c5
I20260809 00:35:11.569607 2061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:35:11.569765 2051 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 00:35:11.573796 2061 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:35:11.574297 2061 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78: Loaded TSK: 0
I20260809 00:35:11.574692 2061 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:35:11.672556 2053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:11.672842 2053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:11.672879 2053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:11.674539 2053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:11.674647 2053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:11.674702 2053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:35:11.676865 2053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.41.212:35027/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:41581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=35645
--tserver_master_addrs=127.21.41.254:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:11.677107 2053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:11.677407 2053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:11.683660 2089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:11.683693 2091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:11.684083 2088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:11.684850 2053 server_base.cc:1034] running on GCE node
I20260809 00:35:11.685068 2053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:11.685375 2053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:11.686553 2053 hybrid_clock.cc:648] HybridClock initialized: now 1786235711686533 us; error 33 us; skew 500 ppm
Aug 09 00:35:11 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:11.690248 2053 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:35:11.690881 2053 webserver.cc:460] Webserver started at http://127.21.41.193:35645/ using document root <none> and password file <none>
I20260809 00:35:11.691232 2053 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:11.691387 2053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:35:11 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:35:11.724570 2053 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260809 00:35:11.730979 2099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.731226 2053 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260809 00:35:11.731313 2053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "06464e21356a4f2fbdbaa3c6197aad3a"
format_stamp: "Formatted at 2026-08-09 00:35:10 on dist-test-slave-zpfg"
server_key: "9e71f8d4484d44aefac4a64babaaf11b"
server_key_iv: "5f27537c3e019a65eb624f90e1dcc45c"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:11.731797 2053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:11.783855 2053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:11.786672 2053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:11.786908 2053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:11.787233 2053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:11.787684 2053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:11.787783 2053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.787890 2053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:11.787968 2053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.796576 2053 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:41581
I20260809 00:35:11.797044 2053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:35:11.797756 2213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:41581 every 8 connection(s)
I20260809 00:35:11.803957 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 2053
I20260809 00:35:11.804333 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:35027/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:45641
--local_ip_for_outbound_sockets=127.21.41.194
--tserver_master_addrs=127.21.41.254:42881
--webserver_port=46067
--webserver_interface=127.21.41.194
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.41.212:35027/kms with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:35:11 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:11.811415 2216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:11.797901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42881 (local address 127.21.41.193:33343)
0809 00:35:11.798063 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:11.798109 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:11.798787 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:11.799577 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:11.799582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:11.799851 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:11.800163 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:11.800169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.802018 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.802019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:11.803353 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:11.803356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.804298 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.804300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:11.804455 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:11.805126 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:11.805137 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:11.807095 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:11.809296 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:11.809303 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:11.809306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:11.809624 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:11.809940 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:11.809941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:11.809942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:11.810084 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:11.810683 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:11.810689 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:11.810857 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 00:35:11.810971 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 00:35:11.811158 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260809 00:35:11.812476 2217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:11.797995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:33343 (local address 127.21.41.254:42881)
0809 00:35:11.799036 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:11.799047 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:11.799086 (+ 39us) server_negotiation.cc:407] Connection header received
0809 00:35:11.799255 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:11.799262 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:11.799328 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:11.799488 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:11.800309 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.801877 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.803452 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:11.804154 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:11.804762 (+ 608us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:11.807227 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:11.807232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:11.807233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:11.807247 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:11.809180 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:11.809733 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:11.809736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:11.809737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:11.809854 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:11.810183 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:11.810184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:11.810185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:11.810555 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:11.810649 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:11.812102 (+ 1453us) server_negotiation.cc:299] Negotiation successful
0809 00:35:11.812257 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":971,"thread_start_us":112,"threads_started":1}
I20260809 00:35:11.813468 2214 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42881
I20260809 00:35:11.813591 2214 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:11.813791 2214 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
W20260809 00:35:11.814790 2214 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:42881 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 00:35:11.814882 2214 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:11.814956 2214 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
I20260809 00:35:11.815337 2214 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:11.815415 2214 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
W20260809 00:35:11.815812 2214 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:35:11.938697 2218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:11.938982 2218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:11.939016 2218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:11.940670 2218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:11.940727 2218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:11.940776 2218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:35:11.942799 2218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.41.212:35027/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:45641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=46067
--tserver_master_addrs=127.21.41.254:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:11.943089 2218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:11.943368 2218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:11.948794 2223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:11.948918 2226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:11.948805 2224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:11.949214 2218 server_base.cc:1034] running on GCE node
I20260809 00:35:11.949426 2218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:11.949716 2218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:11.950904 2218 hybrid_clock.cc:648] HybridClock initialized: now 1786235711950891 us; error 30 us; skew 500 ppm
Aug 09 00:35:11 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:11.954182 2218 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:35:11.954721 2218 webserver.cc:460] Webserver started at http://127.21.41.194:46067/ using document root <none> and password file <none>
I20260809 00:35:11.954988 2218 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:11.955072 2218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:35:11 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:35:11.979190 2218 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:11.980787 2234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.981014 2218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:11.981104 2218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "a8f3fcedf217413e867c09a788cfa765"
format_stamp: "Formatted at 2026-08-09 00:35:10 on dist-test-slave-zpfg"
server_key: "bde5545b437ba04cfa3376bbc8ffb957"
server_key_iv: "2c33d83e59989de934ce4cd559e5637b"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:11.981583 2218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:11.989426 2218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:11.991474 2218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:11.991636 2218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:11.991904 2218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:11.992290 2218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:11.992344 2218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.992386 2218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:11.992413 2218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:11.999622 2218 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:45641
I20260809 00:35:11.999696 2348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:45641 every 8 connection(s)
I20260809 00:35:12.000063 2218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:35:12.004105 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 2218
I20260809 00:35:12.004395 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.41.212:35027/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:43005
--local_ip_for_outbound_sockets=127.21.41.195
--tserver_master_addrs=127.21.41.254:42881
--webserver_port=42889
--webserver_interface=127.21.41.195
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.41.212:35027/kms with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:35:12 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:12.012918 2217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:12.002441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:51997 (local address 127.21.41.254:42881)
0809 00:35:12.002490 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:12.002493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:12.002502 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:35:12.002530 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:12.002532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:12.002544 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:12.002603 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:12.003396 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:12.003974 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:12.006471 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:12.006796 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:12.006844 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:12.009494 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:12.009496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:12.009496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:12.009504 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:12.011301 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:12.011823 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:12.011824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:12.011825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:12.011886 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:12.012161 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:12.012162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:12.012163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:12.012291 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:12.012367 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:12.012711 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 00:35:12.012814 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:35:12.013445 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:12.000870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42881 (local address 127.21.41.194:51997)
0809 00:35:12.001024 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:12.001066 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:12.001524 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:12.002737 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:12.002742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:12.003072 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:12.003289 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:12.003294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:12.004088 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:12.004089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:12.005437 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:12.005440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:12.006961 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:12.006962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:12.007101 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:12.007686 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:12.007696 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:12.009374 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:12.011399 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:12.011406 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:12.011409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:12.011725 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:12.011963 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:12.011963 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:12.011964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:12.012082 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:12.012359 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:12.012366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:12.012538 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 00:35:12.013031 (+ 493us) client_negotiation.cc:240] Negotiation successful
0809 00:35:12.013204 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260809 00:35:12.014015 2349 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42881
I20260809 00:35:12.014146 2349 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:12.014386 2349 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
W20260809 00:35:12.015057 2349 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:42881 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 00:35:12.015142 2349 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:12.015211 2349 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
I20260809 00:35:12.015564 2349 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:12.015687 2349 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
W20260809 00:35:12.015983 2349 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:35:12.127246 2352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:12.127509 2352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:12.127538 2352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:12.128803 2352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:12.128844 2352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:12.128871 2352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:35:12.130679 2352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.41.212:35027/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:43005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=42889
--tserver_master_addrs=127.21.41.254:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:12.130923 2352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:12.131210 2352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:35:12.138258 2352 server_base.cc:1034] running on GCE node
W20260809 00:35:12.138129 2358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:12.138154 2359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:12.138121 2361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:12.138633 2352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:12.138878 2352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:12.140036 2352 hybrid_clock.cc:648] HybridClock initialized: now 1786235712140022 us; error 27 us; skew 500 ppm
Aug 09 00:35:12 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235712, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:12.142550 2352 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:35:12.143049 2352 webserver.cc:460] Webserver started at http://127.21.41.195:42889/ using document root <none> and password file <none>
I20260809 00:35:12.143286 2352 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:12.143342 2352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:35:12 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235712, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:35:12.161760 2352 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:12.162823 2369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:12.163044 2352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:12.163134 2352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "6056018aad3a43d08e9a7204fd08445b"
format_stamp: "Formatted at 2026-08-09 00:35:11 on dist-test-slave-zpfg"
server_key: "14e03fa47c041e6c95ad3c5e617eb1f6"
server_key_iv: "060fc7169014572433391073db356bb5"
server_key_version: "kuduclusterkey@0"
I20260809 00:35:12.163616 2352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:12.185163 2352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:12.187419 2352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:12.187624 2352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:12.187916 2352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:12.188301 2352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:12.188354 2352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:12.188396 2352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:12.188422 2352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:12.195834 2352 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:43005
I20260809 00:35:12.195945 2483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:43005 every 8 connection(s)
I20260809 00:35:12.196406 2352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:35:12.198514 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 2352
Aug 09 00:35:12 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235712, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:12.208076 2217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:12.197203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:57635 (local address 127.21.41.254:42881)
0809 00:35:12.197254 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:12.197256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:12.197483 (+ 227us) server_negotiation.cc:407] Connection header received
0809 00:35:12.198158 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:12.198159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:12.198172 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:12.198215 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:12.199382 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:12.200105 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:12.201569 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:12.201853 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:12.201896 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:12.204506 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:12.204507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:12.204508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:12.204515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:12.206421 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:12.206908 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:12.206910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:12.206910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:12.206968 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:12.207272 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:12.207274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:12.207274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:12.207407 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:12.207449 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:12.207821 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 00:35:12.207970 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":24}
I20260809 00:35:12.208447 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:12.198913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37116 (local address 127.21.41.254:42881)
0809 00:35:12.200393 (+ 1480us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:12.200395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:12.200555 (+ 160us) server_negotiation.cc:407] Connection header received
0809 00:35:12.200658 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:12.200659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:12.200670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:12.200714 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:12.201516 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:12.201990 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:12.203100 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:12.203309 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:12.203389 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:12.204824 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:12.204825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:12.204826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:12.204833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:12.206596 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:12.207004 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:12.207005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:12.207005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:12.207072 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:12.207284 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:12.207285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:12.207285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:12.207422 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:12.207563 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:12.207793 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 00:35:12.207900 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1458,"thread_start_us":58,"threads_started":1}
I20260809 00:35:12.209460 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:12.197183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42881 (local address 127.21.41.195:57635)
0809 00:35:12.197345 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:12.197394 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:12.198030 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:12.198566 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:12.198571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:12.198799 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:12.199268 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:12.199274 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:12.200239 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:12.200239 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:12.201462 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:12.201465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:12.202002 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:12.202003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:12.202147 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:12.202753 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:12.202771 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:12.204395 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:12.206511 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:12.206519 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:12.206521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:12.206813 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:12.207044 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:12.207044 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:12.207045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:12.207174 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:12.207498 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:12.207505 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:12.207656 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 00:35:12.207880 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 00:35:12.209227 (+ 1347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260809 00:35:12.210215 2484 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42881
I20260809 00:35:12.210317 2484 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:12.210538 2484 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
W20260809 00:35:12.211206 2484 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:42881 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 00:35:12.211265 2484 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:12.211308 2484 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
I20260809 00:35:12.211668 2484 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:12.211951 2484 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
W20260809 00:35:12.212373 2484 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 00:35:12 dist-test-slave-zpfg krb5kdc[1009](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786235712, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:12.816002 2214 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:12.816164 2214 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
I20260809 00:35:12.816599 2015 ts_manager.cc:194] Registered new tserver with Master: 06464e21356a4f2fbdbaa3c6197aad3a (127.21.41.193:41581)
I20260809 00:35:12.817632 2015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:33343
I20260809 00:35:13.016201 2349 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:13.016381 2349 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
I20260809 00:35:13.016904 2015 ts_manager.cc:194] Registered new tserver with Master: a8f3fcedf217413e867c09a788cfa765 (127.21.41.194:45641)
I20260809 00:35:13.017464 2015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:51997
Aug 09 00:35:13 dist-test-slave-zpfg krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235712, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for HTTP/127.21.41.212@KRBTEST.COM
I20260809 00:35:13.212570 2484 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:13.212733 2484 heartbeater.cc:511] Master 127.21.41.254:42881 requested a full tablet report, sending...
I20260809 00:35:13.213060 2015 ts_manager.cc:194] Registered new tserver with Master: 6056018aad3a43d08e9a7204fd08445b (127.21.41.195:43005)
I20260809 00:35:13.213598 2015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:57635
I20260809 00:35:13.220052 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:35:13.229583 2506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.221097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.21.41.254:42881)
0809 00:35:13.221219 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:13.221221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:13.221300 (+ 79us) server_negotiation.cc:407] Connection header received
0809 00:35:13.221382 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:13.221383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:13.221395 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:13.221462 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:13.222213 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.223025 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.224202 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.224525 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.224593 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.226318 (+ 1725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:13.226319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:13.226319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:13.226327 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:13.228303 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:13.228672 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:13.228674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:13.228674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:13.228740 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:13.228955 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:13.228956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:13.228956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:13.229075 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:13.229195 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:13.229297 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 00:35:13.229385 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
I20260809 00:35:13.231906 2015 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:35:13.233606 2015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:35:13.241895 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.236404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:41581 (local address 127.0.0.1:51178)
0809 00:35:13.238472 (+ 2068us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:13.238506 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:13.238598 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:13.238883 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:13.238886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:13.238929 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:13.239002 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:13.239005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.239911 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.239912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:13.240722 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:13.240725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.241602 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.241603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:13.241686 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.241699 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 00:35:13.241765 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 00:35:13.241815 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2028,"thread_start_us":90,"threads_started":1}
I20260809 00:35:13.242208 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.236659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51178 (local address 127.21.41.193:41581)
0809 00:35:13.236855 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:13.236861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:13.238588 (+ 1727us) server_negotiation.cc:407] Connection header received
0809 00:35:13.238683 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:13.238685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:13.238706 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:13.238785 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:13.239097 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.239829 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.240830 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.241539 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.241604 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.241663 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:13.242081 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 00:35:13.242135 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":72,"threads_started":1}
I20260809 00:35:13.242928 2145 tablet_service.cc:1467] Processing CreateTablet for tablet d53c451023dd4aadb6ef09edd19e4ad8 (DEFAULT_TABLE table=test-table [id=7ad904dc98d14243a30cb332fb95e411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:13.243345 2145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d53c451023dd4aadb6ef09edd19e4ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:13.246476 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.236713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56246 (local address 127.21.41.194:45641)
0809 00:35:13.236913 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:13.236916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:13.242657 (+ 5741us) server_negotiation.cc:407] Connection header received
0809 00:35:13.242701 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:13.242704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:13.242731 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:13.242812 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:13.243017 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.244011 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.245250 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.245989 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.246048 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.246101 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:13.246323 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 00:35:13.246390 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":58,"threads_started":1}
I20260809 00:35:13.246722 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.236465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:45641 (local address 127.0.0.1:56246)
0809 00:35:13.242310 (+ 5845us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:13.242321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:13.242402 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:13.242900 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:13.242901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:13.242903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:13.242933 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:13.242934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.244114 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.244114 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:13.245137 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:13.245139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.246162 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.246162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:13.246224 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.246227 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:35:13.246594 (+ 367us) client_negotiation.cc:240] Negotiation successful
0809 00:35:13.246638 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5831,"thread_start_us":107,"threads_started":1}
I20260809 00:35:13.247393 2283 tablet_service.cc:1467] Processing CreateTablet for tablet d53c451023dd4aadb6ef09edd19e4ad8 (DEFAULT_TABLE table=test-table [id=7ad904dc98d14243a30cb332fb95e411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:13.247722 2283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d53c451023dd4aadb6ef09edd19e4ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:13.250810 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.236539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54716 (local address 127.21.41.195:43005)
0809 00:35:13.246480 (+ 9941us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:13.246485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:13.246499 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:35:13.246535 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:13.246537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:13.246556 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:13.246652 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:13.246920 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.247918 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.249193 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.249962 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.250014 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.250068 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:13.250680 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 00:35:13.250729 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9899,"thread_start_us":49,"threads_started":1}
I20260809 00:35:13.250947 2512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.236423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:43005 (local address 127.0.0.1:54716)
0809 00:35:13.238723 (+ 2300us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:13.238735 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:13.238773 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:13.246764 (+ 7991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:13.246765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:13.246768 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:13.246831 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:13.246833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.248022 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.248023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:13.249082 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:13.249085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.250133 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.250133 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:13.250206 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.250224 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 00:35:13.250272 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 00:35:13.250310 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2275,"thread_start_us":77,"threads_started":1}
I20260809 00:35:13.251757 2517 tablet_bootstrap.cc:492] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a: Bootstrap starting.
I20260809 00:35:13.251714 2401 tablet_service.cc:1467] Processing CreateTablet for tablet d53c451023dd4aadb6ef09edd19e4ad8 (DEFAULT_TABLE table=test-table [id=7ad904dc98d14243a30cb332fb95e411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:13.252079 2401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d53c451023dd4aadb6ef09edd19e4ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:13.252439 2517 tablet_bootstrap.cc:654] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:13.252748 2517 log.cc:824] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:13.253531 2517 tablet_bootstrap.cc:492] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a: No bootstrap required, opened a new log
I20260809 00:35:13.253623 2517 ts_tablet_manager.cc:1397] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:35:13.255679 2520 tablet_bootstrap.cc:492] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b: Bootstrap starting.
I20260809 00:35:13.255570 2517 raft_consensus.cc:348] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41581 } } peers { permanent_uuid: "6056018aad3a43d08e9a7204fd08445b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43005 } } peers { permanent_uuid: "a8f3fcedf217413e867c09a788cfa765" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45641 } }
I20260809 00:35:13.255765 2517 raft_consensus.cc:374] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:13.255823 2517 raft_consensus.cc:729] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06464e21356a4f2fbdbaa3c6197aad3a, State: Initialized, Role: FOLLOWER
I20260809 00:35:13.256021 2517 consensus_queue.cc:260] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41581 } } peers { permanent_uuid: "6056018aad3a43d08e9a7204fd08445b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43005 } } peers { permanent_uuid: "a8f3fcedf217413e867c09a788cfa765" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45641 } }
I20260809 00:35:13.256405 2517 ts_tablet_manager.cc:1428] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:35:13.256481 2520 tablet_bootstrap.cc:654] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:13.256592 2214 heartbeater.cc:503] Master 127.21.41.254:42881 was elected leader, sending a full tablet report...
I20260809 00:35:13.256817 2520 log.cc:824] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:13.257901 2520 tablet_bootstrap.cc:492] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b: No bootstrap required, opened a new log
I20260809 00:35:13.257992 2520 ts_tablet_manager.cc:1397] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:35:13.258569 2518 tablet_bootstrap.cc:492] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765: Bootstrap starting.
I20260809 00:35:13.259346 2518 tablet_bootstrap.cc:654] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:13.259722 2518 log.cc:824] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:13.260712 2520 raft_consensus.cc:348] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41581 } } peers { permanent_uuid: "6056018aad3a43d08e9a7204fd08445b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43005 } } peers { permanent_uuid: "a8f3fcedf217413e867c09a788cfa765" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45641 } }
I20260809 00:35:13.260864 2520 raft_consensus.cc:374] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:13.260937 2520 raft_consensus.cc:729] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6056018aad3a43d08e9a7204fd08445b, State: Initialized, Role: FOLLOWER
I20260809 00:35:13.261083 2520 consensus_queue.cc:260] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41581 } } peers { permanent_uuid: "6056018aad3a43d08e9a7204fd08445b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43005 } } peers { permanent_uuid: "a8f3fcedf217413e867c09a788cfa765" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45641 } }
I20260809 00:35:13.261369 2520 ts_tablet_manager.cc:1428] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:35:13.261408 2484 heartbeater.cc:503] Master 127.21.41.254:42881 was elected leader, sending a full tablet report...
I20260809 00:35:13.267350 2518 tablet_bootstrap.cc:492] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765: No bootstrap required, opened a new log
I20260809 00:35:13.267447 2518 ts_tablet_manager.cc:1397] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260809 00:35:13.269821 2518 raft_consensus.cc:348] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41581 } } peers { permanent_uuid: "6056018aad3a43d08e9a7204fd08445b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43005 } } peers { permanent_uuid: "a8f3fcedf217413e867c09a788cfa765" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45641 } }
I20260809 00:35:13.269954 2518 raft_consensus.cc:374] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:13.269976 2518 raft_consensus.cc:729] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8f3fcedf217413e867c09a788cfa765, State: Initialized, Role: FOLLOWER
I20260809 00:35:13.270097 2518 consensus_queue.cc:260] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41581 } } peers { permanent_uuid: "6056018aad3a43d08e9a7204fd08445b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43005 } } peers { permanent_uuid: "a8f3fcedf217413e867c09a788cfa765" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45641 } }
I20260809 00:35:13.270407 2349 heartbeater.cc:503] Master 127.21.41.254:42881 was elected leader, sending a full tablet report...
I20260809 00:35:13.270407 2518 ts_tablet_manager.cc:1428] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260809 00:35:13.298591 2215 tablet.cc:2369] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:35:13.326586 2525 raft_consensus.cc:482] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:35:13.326912 2525 raft_consensus.cc:504] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41581 } } peers { permanent_uuid: "6056018aad3a43d08e9a7204fd08445b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43005 } } peers { permanent_uuid: "a8f3fcedf217413e867c09a788cfa765" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45641 } }
I20260809 00:35:13.327965 2525 leader_election.cc:288] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06464e21356a4f2fbdbaa3c6197aad3a (127.21.41.193:41581), 6056018aad3a43d08e9a7204fd08445b (127.21.41.195:43005)
I20260809 00:35:13.332617 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.327890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:39957 (local address 127.21.41.195:43005)
0809 00:35:13.327940 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:13.327942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:13.328600 (+ 658us) server_negotiation.cc:407] Connection header received
0809 00:35:13.328747 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:13.328749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:13.328760 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:13.328796 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:13.329044 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.329817 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.331513 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.332150 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.332206 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.332225 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:13.332483 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 00:35:13.332529 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:35:13.333133 2526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.327648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:43005 (local address 127.21.41.194:39957)
0809 00:35:13.328523 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:13.328535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:13.328650 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:13.328873 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:13.328874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:13.328876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:13.328962 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:13.328964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.329952 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.329952 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:13.331339 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:13.331342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.332316 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.332317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:13.332388 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.332392 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:35:13.332954 (+ 562us) client_negotiation.cc:240] Negotiation successful
0809 00:35:13.333005 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"thread_start_us":108,"threads_started":1}
I20260809 00:35:13.333642 2438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d53c451023dd4aadb6ef09edd19e4ad8" candidate_uuid: "a8f3fcedf217413e867c09a788cfa765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6056018aad3a43d08e9a7204fd08445b" is_pre_election: true
I20260809 00:35:13.333827 2438 raft_consensus.cc:2454] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8f3fcedf217413e867c09a788cfa765 in term 0.
I20260809 00:35:13.334172 2237 leader_election.cc:302] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6056018aad3a43d08e9a7204fd08445b, a8f3fcedf217413e867c09a788cfa765; no voters:
I20260809 00:35:13.334514 2525 raft_consensus.cc:2781] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:35:13.334592 2525 raft_consensus.cc:482] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:35:13.334630 2525 raft_consensus.cc:3037] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:13.335632 2525 raft_consensus.cc:504] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41581 } } peers { permanent_uuid: "6056018aad3a43d08e9a7204fd08445b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43005 } } peers { permanent_uuid: "a8f3fcedf217413e867c09a788cfa765" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45641 } }
I20260809 00:35:13.336056 2438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d53c451023dd4aadb6ef09edd19e4ad8" candidate_uuid: "a8f3fcedf217413e867c09a788cfa765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6056018aad3a43d08e9a7204fd08445b"
I20260809 00:35:13.336143 2438 raft_consensus.cc:3037] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:13.336550 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.330059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:57949 (local address 127.21.41.193:41581)
0809 00:35:13.330115 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:13.330118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:13.332818 (+ 2700us) server_negotiation.cc:407] Connection header received
0809 00:35:13.332927 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:13.332928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:13.332938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:13.332973 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:13.333197 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.333931 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.335362 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.336039 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.336089 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.336097 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:13.336434 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 00:35:13.336479 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:35:13.336786 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.328235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:41581 (local address 127.21.41.194:57949)
0809 00:35:13.332733 (+ 4498us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:13.332748 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:13.332850 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:13.333039 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:13.333039 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:13.333041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:13.333096 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:13.333097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.334035 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.334036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:13.335243 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:13.335246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.336235 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.336236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:13.336327 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.336331 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:35:13.336658 (+ 327us) client_negotiation.cc:240] Negotiation successful
0809 00:35:13.336699 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4473,"thread_start_us":88,"threads_started":1}
I20260809 00:35:13.337196 2438 raft_consensus.cc:2454] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8f3fcedf217413e867c09a788cfa765 in term 1.
I20260809 00:35:13.337185 2165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d53c451023dd4aadb6ef09edd19e4ad8" candidate_uuid: "a8f3fcedf217413e867c09a788cfa765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" is_pre_election: true
I20260809 00:35:13.337364 2165 raft_consensus.cc:2454] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8f3fcedf217413e867c09a788cfa765 in term 0.
I20260809 00:35:13.337494 2155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d53c451023dd4aadb6ef09edd19e4ad8" candidate_uuid: "a8f3fcedf217413e867c09a788cfa765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06464e21356a4f2fbdbaa3c6197aad3a"
I20260809 00:35:13.337568 2155 raft_consensus.cc:3037] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:13.337751 2237 leader_election.cc:302] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6056018aad3a43d08e9a7204fd08445b, a8f3fcedf217413e867c09a788cfa765; no voters:
I20260809 00:35:13.337978 2525 leader_election.cc:288] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [CANDIDATE]: Term 1 election: Requested vote from peers 06464e21356a4f2fbdbaa3c6197aad3a (127.21.41.193:41581), 6056018aad3a43d08e9a7204fd08445b (127.21.41.195:43005)
I20260809 00:35:13.338065 2525 raft_consensus.cc:2781] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:35:13.338147 2525 raft_consensus.cc:686] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 1 LEADER]: Becoming Leader. State: Replica: a8f3fcedf217413e867c09a788cfa765, State: Running, Role: LEADER
I20260809 00:35:13.338289 2525 consensus_queue.cc:237] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41581 } } peers { permanent_uuid: "6056018aad3a43d08e9a7204fd08445b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43005 } } peers { permanent_uuid: "a8f3fcedf217413e867c09a788cfa765" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45641 } }
I20260809 00:35:13.338567 2155 raft_consensus.cc:2454] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8f3fcedf217413e867c09a788cfa765 in term 1.
I20260809 00:35:13.339409 2015 catalog_manager.cc:5495] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8f3fcedf217413e867c09a788cfa765 (127.21.41.194). New cstate: current_term: 1 leader_uuid: "a8f3fcedf217413e867c09a788cfa765" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6056018aad3a43d08e9a7204fd08445b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8f3fcedf217413e867c09a788cfa765" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 45641 } health_report { overall_health: HEALTHY } } }
I20260809 00:35:13.359505 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.355967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56262 (local address 127.21.41.194:45641)
0809 00:35:13.356019 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:13.356021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:13.356072 (+ 51us) server_negotiation.cc:407] Connection header received
0809 00:35:13.356167 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:13.356169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:13.356191 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:13.356293 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:35:13.356450 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.356991 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.358210 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:13.358558 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:13.358672 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:13.358901 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:35:13.359047 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:35:13.359138 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:13.359398 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 00:35:13.359421 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:35:13.362214 2155 raft_consensus.cc:1261] T d53c451023dd4aadb6ef09edd19e4ad8 P 06464e21356a4f2fbdbaa3c6197aad3a [term 1 FOLLOWER]: Refusing update from remote peer a8f3fcedf217413e867c09a788cfa765: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:13.362214 2438 raft_consensus.cc:1261] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b [term 1 FOLLOWER]: Refusing update from remote peer a8f3fcedf217413e867c09a788cfa765: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:13.362684 2525 consensus_queue.cc:1035] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6056018aad3a43d08e9a7204fd08445b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:13.362946 2525 consensus_queue.cc:1035] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06464e21356a4f2fbdbaa3c6197aad3a" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:13.369678 2534 mvcc.cc:205] Tried to move back new op lower bound from 7316421481926709248 to 7316421481835077632. Current Snapshot: MvccSnapshot[applied={T|T < 7316421481926709248}]
I20260809 00:35:13.372552 2532 mvcc.cc:205] Tried to move back new op lower bound from 7316421481926709248 to 7316421481835077632. Current Snapshot: MvccSnapshot[applied={T|T < 7316421481926709248}]
I20260809 00:35:13.377200 2015 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:35:13.377337 2015 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:35:13.378260 2015 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90be3d7a5bb64a039433024bfafa1c78: Sending DeleteTablet for 3 replicas of tablet d53c451023dd4aadb6ef09edd19e4ad8
I20260809 00:35:13.378716 2401 tablet_service.cc:1514] Processing DeleteTablet for tablet d53c451023dd4aadb6ef09edd19e4ad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:13 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:54716
I20260809 00:35:13.378762 2145 tablet_service.cc:1514] Processing DeleteTablet for tablet d53c451023dd4aadb6ef09edd19e4ad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:13 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:51178
I20260809 00:35:13.379037 2541 tablet_replica.cc:330] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b: stopping tablet replica
I20260809 00:35:13.379038 2283 tablet_service.cc:1514] Processing DeleteTablet for tablet d53c451023dd4aadb6ef09edd19e4ad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:13 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:56246
I20260809 00:35:13.379172 2541 raft_consensus.cc:2229] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:35:13.379299 2541 raft_consensus.cc:2258] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:35:13.379874 2541 ts_tablet_manager.cc:1905] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:35:13.379938 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 2053
I20260809 00:35:13.383893 2541 ts_tablet_manager.cc:1918] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:35:13.384070 2541 log.cc:1196] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/d53c451023dd4aadb6ef09edd19e4ad8
I20260809 00:35:13.384508 2541 ts_tablet_manager.cc:1939] T d53c451023dd4aadb6ef09edd19e4ad8 P 6056018aad3a43d08e9a7204fd08445b: Deleting consensus metadata
I20260809 00:35:13.384843 2543 tablet_replica.cc:330] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765: stopping tablet replica
I20260809 00:35:13.385001 2543 raft_consensus.cc:2229] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:35:13.385182 2543 raft_consensus.cc:2258] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:35:13.385236 2001 catalog_manager.cc:4841] TS 6056018aad3a43d08e9a7204fd08445b (127.21.41.195:43005): tablet d53c451023dd4aadb6ef09edd19e4ad8 (table test-table [id=7ad904dc98d14243a30cb332fb95e411]) successfully deleted
I20260809 00:35:13.385740 2543 ts_tablet_manager.cc:1905] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:35:13.387566 2543 ts_tablet_manager.cc:1918] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:35:13.391460 2543 log.cc:1196] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/d53c451023dd4aadb6ef09edd19e4ad8
I20260809 00:35:13.391939 2543 ts_tablet_manager.cc:1939] T d53c451023dd4aadb6ef09edd19e4ad8 P a8f3fcedf217413e867c09a788cfa765: Deleting consensus metadata
I20260809 00:35:13.392457 2002 catalog_manager.cc:4841] TS a8f3fcedf217413e867c09a788cfa765 (127.21.41.194:45641): tablet d53c451023dd4aadb6ef09edd19e4ad8 (table test-table [id=7ad904dc98d14243a30cb332fb95e411]) successfully deleted
W20260809 00:35:13.392750 1999 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:41581 (error 108)
I20260809 00:35:13.393805 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:13.393661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:41581 (local address 127.0.0.1:51184)
0809 00:35:13.393717 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:13.393759 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 00:35:13.394028 1999 catalog_manager.cc:4584] TS 06464e21356a4f2fbdbaa3c6197aad3a (127.21.41.193:41581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d53c451023dd4aadb6ef09edd19e4ad8: Network error: Client connection negotiation failed: client connection to 127.21.41.193:41581: connect: Connection refused (error 111)
I20260809 00:35:13.394302 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 2218
I20260809 00:35:13.400970 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 2352
I20260809 00:35:13.407846 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 1982
I20260809 00:35:13.414057 21671 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 00:35:13.759114 21671 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 00:35:13.759253 21671 mini_ranger.cc:66] Stopping Ranger...
I20260809 00:35:14.104115 21671 mini_ranger.cc:68] Stopped Ranger
2026-08-09 00:35:14.104 UTC [1039] LOG: received smart shutdown request
2026-08-09 00:35:14.108 UTC [1039] LOG: background worker "logical replication launcher" (PID 1048) exited with exit code 1
2026-08-09 00:35:14.108 UTC [1043] LOG: shutting down
2026-08-09 00:35:14.282 UTC [1039] LOG: database system is shut down
2026-08-09T00:35:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49039 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:35:14 dist-test-slave-zpfg krb5kdc[2554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:35:14 dist-test-slave-zpfg krb5kdc[2554](info): set up 2 sockets
Aug 09 00:35:14 dist-test-slave-zpfg krb5kdc[2554](info): commencing operation
krb5kdc: starting...
W20260809 00:35:15.435881 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:35:15 dist-test-slave-zpfg krb5kdc[2554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:35:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:35:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:35:15.557884 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:41985
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:41985,127.21.41.253:35089,127.21.41.252:38335
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:15.645906 2567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:15.646166 2567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:15.646205 2567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:15.647480 2567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:35:15.647550 2567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:15.647575 2567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:15.647605 2567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:35:15.647632 2567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:35:15.649209 2567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:41985,127.21.41.253:35089,127.21.41.252:38335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:41985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:15.649492 2567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:15.649735 2567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:35:15.654718 2567 server_base.cc:1034] running on GCE node
W20260809 00:35:15.654871 2573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:15.654893 2572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:15.654951 2575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:15.655356 2567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:15.655706 2567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:15.656876 2567 hybrid_clock.cc:648] HybridClock initialized: now 1786235715656865 us; error 31 us; skew 500 ppm
Aug 09 00:35:15 dist-test-slave-zpfg krb5kdc[2554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:15.661868 2567 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:35:15.662410 2567 webserver.cc:460] Webserver started at http://127.21.41.254:36691/ using document root <none> and password file <none>
I20260809 00:35:15.662652 2567 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:15.662720 2567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:15.662874 2567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:15.663945 2567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff"
format_stamp: "Formatted at 2026-08-09 00:35:15 on dist-test-slave-zpfg"
server_key: "eaa64c85fd9ed704d9eeba51cc3ee18a"
server_key_iv: "0174e51cfe48ce87fe986bfbe5c2e57e"
server_key_version: "encryptionkey@0"
I20260809 00:35:15.664302 2567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff"
format_stamp: "Formatted at 2026-08-09 00:35:15 on dist-test-slave-zpfg"
server_key: "eaa64c85fd9ed704d9eeba51cc3ee18a"
server_key_iv: "0174e51cfe48ce87fe986bfbe5c2e57e"
server_key_version: "encryptionkey@0"
I20260809 00:35:15.665640 2567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:35:15.666392 2582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:15.666574 2567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:15.666675 2567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff"
format_stamp: "Formatted at 2026-08-09 00:35:15 on dist-test-slave-zpfg"
server_key: "eaa64c85fd9ed704d9eeba51cc3ee18a"
server_key_iv: "0174e51cfe48ce87fe986bfbe5c2e57e"
server_key_version: "encryptionkey@0"
I20260809 00:35:15.666761 2567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:15.677901 2567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:15.680177 2567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:15.680346 2567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:15.684394 2567 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:41985
I20260809 00:35:15.684427 2634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:41985 every 8 connection(s)
I20260809 00:35:15.684773 2567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:35:15.685614 2635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:15.687233 2635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:35:15.692705 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 2567
I20260809 00:35:15.692786 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:35:15.692977 21671 external_mini_cluster.cc:1428] Setting key c08c66afd7b4fd2ef3c4907be614cba0
Aug 09 00:35:15 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:15.697165 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.687920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:41985 (local address 127.0.0.1:47932)
0809 00:35:15.688123 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.688155 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:15.688591 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:15.688937 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:15.688940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:15.689218 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:15.689382 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:15.689386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.690401 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.690404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:15.691084 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:15.691085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.691567 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.691568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:15.691622 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:15.692058 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:15.692066 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:15.693514 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:15.695702 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:15.695708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:15.695710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:15.695901 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:15.696147 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:15.696147 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:15.696148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:15.696242 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:15.696586 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:15.696591 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:15.696651 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 00:35:15.696760 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 00:35:15.696987 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":91,"thread_start_us":111,"threads_started":1}
I20260809 00:35:15.697177 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.687926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.21.41.254:41985)
0809 00:35:15.688084 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:15.688093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:15.688246 (+ 153us) server_negotiation.cc:407] Connection header received
0809 00:35:15.688801 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:15.688803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:15.688820 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:15.688907 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:15.689466 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.690273 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.691171 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.691488 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.691580 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:15.693603 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:15.693611 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:15.693611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:15.693618 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:15.695586 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:15.695997 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:15.696000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:15.696001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:15.696078 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:15.696307 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:15.696308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:15.696309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:15.696515 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:15.696606 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:15.696787 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 00:35:15.696924 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260809 00:35:15.697868 2635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:35:15.698251 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.698099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35089 (local address 127.0.0.1:34146)
0809 00:35:15.698156 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.698213 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 00:35:15.698447 2584 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.41.253:35089: connect: Connection refused (error 111)
I20260809 00:35:15.698939 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.698827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35089 (local address 127.0.0.1:34156)
0809 00:35:15.698879 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.698912 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 00:35:15.699072 2635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.253:35089: Network error: Client connection negotiation failed: client connection to 127.21.41.253:35089: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.21.41.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab.
Entry for principal kudu/127.21.41.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab.
I20260809 00:35:15.720924 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.253:35089
--webserver_interface=127.21.41.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab
--principal=kudu/127.21.41.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:41985,127.21.41.253:35089,127.21.41.252:38335
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:35:15.748328 2635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } attempt: 1
I20260809 00:35:15.748893 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.748738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35089 (local address 127.0.0.1:34168)
0809 00:35:15.748805 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.748856 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:35:15.749264 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.749171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35089 (local address 127.0.0.1:34180)
0809 00:35:15.749216 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.749244 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:35:15.749382 2635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.253:35089: Network error: Client connection negotiation failed: client connection to 127.21.41.253:35089: connect: Connection refused (error 111)
W20260809 00:35:15.812248 2642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:15.812446 2642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:15.812469 2642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:15.813694 2642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:35:15.813737 2642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:15.813752 2642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:15.813803 2642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:35:15.813844 2642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:35:15.815414 2642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:41985,127.21.41.253:35089,127.21.41.252:38335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab
--principal=kudu/127.21.41.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.253:35089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.41.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:15.815680 2642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:15.815922 2642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:35:15.817559 2635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } attempt: 2
I20260809 00:35:15.817991 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.817841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35089 (local address 127.0.0.1:34196)
0809 00:35:15.817908 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.817953 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:35:15.818380 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.818256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35089 (local address 127.0.0.1:34212)
0809 00:35:15.818300 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.818327 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:35:15.818481 2635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.253:35089: Network error: Client connection negotiation failed: client connection to 127.21.41.253:35089: connect: Connection refused (error 111)
W20260809 00:35:15.820878 2647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:15.820902 2648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:15.820932 2650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:15.821125 2642 server_base.cc:1034] running on GCE node
I20260809 00:35:15.821377 2642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:15.821643 2642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:15.822739 2642 hybrid_clock.cc:648] HybridClock initialized: now 1786235715822716 us; error 34 us; skew 500 ppm
Aug 09 00:35:15 dist-test-slave-zpfg krb5kdc[2554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:15.825170 2642 init.cc:377] Logged in from keytab as kudu/127.21.41.253@KRBTEST.COM (short username kudu)
I20260809 00:35:15.825533 2642 webserver.cc:460] Webserver started at http://127.21.41.253:39819/ using document root <none> and password file <none>
I20260809 00:35:15.825706 2642 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:15.825784 2642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:15.825906 2642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:15.826947 2642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/data/instance:
uuid: "3cb2781858c441dabf6730d3a62319b6"
format_stamp: "Formatted at 2026-08-09 00:35:15 on dist-test-slave-zpfg"
server_key: "ddde4804904276dcf91759e1abfc1724"
server_key_iv: "976a08bc7c6599d1ba079d8b174353ba"
server_key_version: "encryptionkey@0"
I20260809 00:35:15.827266 2642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/wal/instance:
uuid: "3cb2781858c441dabf6730d3a62319b6"
format_stamp: "Formatted at 2026-08-09 00:35:15 on dist-test-slave-zpfg"
server_key: "ddde4804904276dcf91759e1abfc1724"
server_key_iv: "976a08bc7c6599d1ba079d8b174353ba"
server_key_version: "encryptionkey@0"
I20260809 00:35:15.828511 2642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:35:15.829152 2657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:15.829344 2642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:35:15.829412 2642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/wal
uuid: "3cb2781858c441dabf6730d3a62319b6"
format_stamp: "Formatted at 2026-08-09 00:35:15 on dist-test-slave-zpfg"
server_key: "ddde4804904276dcf91759e1abfc1724"
server_key_iv: "976a08bc7c6599d1ba079d8b174353ba"
server_key_version: "encryptionkey@0"
I20260809 00:35:15.829463 2642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:15.846207 2642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:15.848253 2642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:15.848376 2642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:15.852463 2642 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.253:35089
I20260809 00:35:15.852496 2709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.253:35089 every 8 connection(s)
I20260809 00:35:15.852843 2642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
I20260809 00:35:15.853731 2710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:15.855269 2710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:35:15.856317 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 2642
I20260809 00:35:15.856401 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-1/wal/instance
I20260809 00:35:15.856582 21671 external_mini_cluster.cc:1428] Setting key f7f4622eba685cf6d33d73cb81d63d0e
Aug 09 00:35:15 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.41.252@KRBTEST.COM; defaulting to no policy
I20260809 00:35:15.865468 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.856005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47946 (local address 127.21.41.254:41985)
0809 00:35:15.856066 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:15.856067 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:15.856276 (+ 209us) server_negotiation.cc:407] Connection header received
0809 00:35:15.856798 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:15.856800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:15.856813 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:15.856890 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:15.857509 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.858279 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.859645 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.859995 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.860073 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:15.862487 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:15.862488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:15.862489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:15.862498 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:15.864049 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:15.864459 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:15.864461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:15.864461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:15.864527 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:15.864777 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:15.864778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:15.864779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:15.864925 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:15.865013 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:15.865267 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 00:35:15.865363 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:35:15.865551 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.855969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:41985 (local address 127.0.0.1:47946)
0809 00:35:15.856139 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.856186 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:15.856699 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:15.856957 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:15.856960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:15.857213 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:15.857416 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:15.857420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.858405 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.858406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:15.859557 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:15.859559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.860080 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.860081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:15.860180 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:15.860742 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:15.860753 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:15.862381 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:15.864137 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:15.864144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:15.864146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:15.864376 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:15.864612 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:15.864612 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:15.864612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:15.864707 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:15.865000 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:15.865004 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:15.865111 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 00:35:15.865190 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:35:15.865342 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260809 00:35:15.866165 2710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.252@KRBTEST.COM" created.
Aug 09 00:35:15 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
I20260809 00:35:15.873920 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.866456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35089 (local address 127.0.0.1:34222)
0809 00:35:15.866511 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.866522 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:15.866568 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:15.866826 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:15.866827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:15.866961 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:15.867002 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:15.867004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.868032 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.868033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:15.868595 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:15.868597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.869300 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.869301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:15.869376 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:15.869814 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:15.869820 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:15.871102 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:15.872871 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:15.872872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:15.872872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:15.873039 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:15.873257 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:15.873258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:15.873258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:15.873312 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:15.873600 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:15.873600 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:15.873637 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 00:35:15.873698 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:35:15.873797 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 00:35:15.873952 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.866529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34222 (local address 127.21.41.253:35089)
0809 00:35:15.866684 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:15.866688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:15.866703 (+ 15us) server_negotiation.cc:407] Connection header received
0809 00:35:15.866718 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:15.866720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:15.866730 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:15.866811 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:15.867085 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.867938 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.868669 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.869183 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.869254 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:15.871203 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:15.871211 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:15.871212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:15.871218 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:15.872761 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:15.873133 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:15.873136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:15.873136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:15.873191 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:15.873395 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:15.873396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:15.873396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:15.873530 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:15.873622 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:15.873736 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 00:35:15.873849 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260809 00:35:15.874562 2710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:35:15.874954 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.874829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43842)
0809 00:35:15.874880 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.874922 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 00:35:15.875108 2661 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
I20260809 00:35:15.875527 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.875438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43858)
0809 00:35:15.875481 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.875505 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 00:35:15.875650 2710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:38335: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab.
Entry for principal kudu/127.21.41.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab.
I20260809 00:35:15.886475 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.252:38335
--webserver_interface=127.21.41.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab
--principal=kudu/127.21.41.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:41985,127.21.41.253:35089,127.21.41.252:38335
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:35:15.909659 2635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } attempt: 3
Aug 09 00:35:15 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
I20260809 00:35:15.916977 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.910147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.21.41.253:35089)
0809 00:35:15.910215 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:15.910218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:15.910231 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:35:15.910281 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:15.910282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:15.910295 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:15.910395 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:15.910725 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.911252 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.911987 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.912201 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.912282 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:15.914537 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:15.914539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:15.914539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:15.914545 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:15.915863 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:15.916278 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:15.916279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:15.916279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:15.916331 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:15.916531 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:15.916532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:15.916532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:15.916633 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:15.916681 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:15.916847 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 00:35:15.916908 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:35:15.917029 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.910055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35089 (local address 127.0.0.1:34224)
0809 00:35:15.910122 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.910134 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:15.910206 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:15.910405 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:15.910406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:15.910567 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:15.910631 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:15.910633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.911347 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.911348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:15.911900 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:15.911902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:15.912291 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:15.912292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:15.912392 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:15.912927 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:15.912932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:15.914453 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:15.915960 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:15.915961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:15.915962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:15.916201 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:15.916401 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:15.916402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:15.916403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:15.916472 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:15.916693 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:15.916694 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:15.916746 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 00:35:15.916832 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:35:15.916934 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:35:15.917553 2635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:35:15.917920 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.917810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43862)
0809 00:35:15.917858 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.917894 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 00:35:15.918313 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.918211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43870)
0809 00:35:15.918269 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.918293 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 00:35:15.918428 2635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:38335: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
I20260809 00:35:15.924883 2710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } attempt: 1
I20260809 00:35:15.925355 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.925206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43882)
0809 00:35:15.925272 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.925314 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:35:15.925727 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.925627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43892)
0809 00:35:15.925675 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.925702 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 00:35:15.925834 2710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:38335: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
I20260809 00:35:15.949584 2635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } attempt: 1
I20260809 00:35:15.950130 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.949991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43896)
0809 00:35:15.950060 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.950101 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 00:35:15.950546 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.950449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43910)
0809 00:35:15.950495 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.950525 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:35:15.950665 2635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:38335: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
W20260809 00:35:15.970980 2717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:15.971184 2717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:15.971206 2717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:15.972339 2717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:35:15.972380 2717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:15.972394 2717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:15.972409 2717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:35:15.972455 2717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:35:15.973948 2717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:41985,127.21.41.253:35089,127.21.41.252:38335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab
--principal=kudu/127.21.41.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.252:38335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.41.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:15.974205 2717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:15.974488 2717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:15.979547 2725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:15.979550 2722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:15.979588 2717 server_base.cc:1034] running on GCE node
W20260809 00:35:15.979548 2723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:15.979954 2717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:15.980247 2717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:15.981415 2717 hybrid_clock.cc:648] HybridClock initialized: now 1786235715981413 us; error 39 us; skew 500 ppm
Aug 09 00:35:15 dist-test-slave-zpfg krb5kdc[2554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:15.983945 2717 init.cc:377] Logged in from keytab as kudu/127.21.41.252@KRBTEST.COM (short username kudu)
I20260809 00:35:15.984431 2717 webserver.cc:460] Webserver started at http://127.21.41.252:43845/ using document root <none> and password file <none>
I20260809 00:35:15.984654 2717 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:15.984725 2717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:15.984884 2717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:15.985942 2717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/data/instance:
uuid: "a7cfe89d74a1476f96f16b87958653c9"
format_stamp: "Formatted at 2026-08-09 00:35:15 on dist-test-slave-zpfg"
server_key: "2194dd987a296f0a58aa46592979b1fc"
server_key_iv: "85cc10f2dbd953856d5e71d5a0ab61df"
server_key_version: "encryptionkey@0"
I20260809 00:35:15.986308 2717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/wal/instance:
uuid: "a7cfe89d74a1476f96f16b87958653c9"
format_stamp: "Formatted at 2026-08-09 00:35:15 on dist-test-slave-zpfg"
server_key: "2194dd987a296f0a58aa46592979b1fc"
server_key_iv: "85cc10f2dbd953856d5e71d5a0ab61df"
server_key_version: "encryptionkey@0"
I20260809 00:35:15.987630 2717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:15.988540 2732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:15.988722 2717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:15.988829 2717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/wal
uuid: "a7cfe89d74a1476f96f16b87958653c9"
format_stamp: "Formatted at 2026-08-09 00:35:15 on dist-test-slave-zpfg"
server_key: "2194dd987a296f0a58aa46592979b1fc"
server_key_iv: "85cc10f2dbd953856d5e71d5a0ab61df"
server_key_version: "encryptionkey@0"
I20260809 00:35:15.988919 2717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:15.993975 2710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } attempt: 2
I20260809 00:35:15.994493 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.994308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43916)
0809 00:35:15.994420 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.994460 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 00:35:15.994863 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:15.994747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43932)
0809 00:35:15.994807 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:15.994842 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 00:35:15.994958 2710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:38335: Network error: Client connection negotiation failed: client connection to 127.21.41.252:38335: connect: Connection refused (error 111)
I20260809 00:35:16.017745 2717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:16.020027 2717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:16.020223 2717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:16.024984 2717 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.252:38335
I20260809 00:35:16.025022 2784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.252:38335 every 8 connection(s)
I20260809 00:35:16.025445 2717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
I20260809 00:35:16.025846 2635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } attempt: 2
I20260809 00:35:16.026584 2785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:16.028417 2785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:35:16.031504 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 2717
I20260809 00:35:16.031620 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786235559853195-21671-0/minicluster-data/master-2/wal/instance
I20260809 00:35:16.031841 21671 external_mini_cluster.cc:1428] Setting key 0bbef7b25003452072806c7303539bd6
Aug 09 00:35:16 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
Aug 09 00:35:16 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:16.036955 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.026203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43946)
0809 00:35:16.026270 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:16.026283 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:16.026365 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:16.027447 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:16.027448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:16.027649 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:16.027701 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:16.027704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.029070 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.029071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:16.029862 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:16.029866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.030587 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.030588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:16.030651 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.031139 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:16.031144 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:16.032845 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:16.035207 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:16.035208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:16.035208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:16.035413 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:16.036020 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:16.036021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:16.036021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:16.036116 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:16.036573 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:16.036574 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:16.036625 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 00:35:16.036730 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 00:35:16.036838 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:35:16.037227 2787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.026417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43946 (local address 127.21.41.252:38335)
0809 00:35:16.026666 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:16.026678 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:16.026718 (+ 40us) server_negotiation.cc:407] Connection header received
0809 00:35:16.027231 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:16.027239 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:16.027292 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:16.027463 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:16.027857 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.028952 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.029954 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.030489 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.030636 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.032951 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:16.032956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:16.032957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:16.032963 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:16.035133 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.035824 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.035828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.035828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.035946 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.036203 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.036204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.036205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.036471 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:16.036580 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:16.036804 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 00:35:16.036936 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":90,"threads_started":1}
Aug 09 00:35:16 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
I20260809 00:35:16.039108 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.029115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47958 (local address 127.21.41.254:41985)
0809 00:35:16.029170 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:16.029173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:16.029386 (+ 213us) server_negotiation.cc:407] Connection header received
0809 00:35:16.029446 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:16.029446 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:16.029456 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:16.029501 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:16.030094 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.030627 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.031809 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.032162 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.032239 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.034967 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:16.034969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:16.034969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:16.034977 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:16.037561 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.038051 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.038052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.038052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.038119 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.038418 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.038419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.038419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.038592 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:16.038637 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:16.038910 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 00:35:16.039005 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 00:35:16.039304 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.029083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:41985 (local address 127.0.0.1:47958)
0809 00:35:16.029259 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:16.029303 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:16.029387 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:16.029532 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:16.029535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:16.029911 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:16.030001 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:16.030004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.030722 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.030723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:16.031654 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:16.031657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.032308 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.032310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:16.032407 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.032902 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:16.032913 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:16.034865 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:16.037707 (+ 2842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:16.037714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:16.037716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:16.037970 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:16.038224 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:16.038225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:16.038226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:16.038300 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:16.038692 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:16.038699 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:16.038754 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 00:35:16.038853 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 00:35:16.038978 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260809 00:35:16.040086 2785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:35:16.040117 2635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff: Bootstrap starting.
I20260809 00:35:16.041023 2635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:16.041409 2635 log.cc:824] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:16.041610 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.032196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.21.41.252:38335)
0809 00:35:16.032357 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:16.032364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:16.032432 (+ 68us) server_negotiation.cc:407] Connection header received
0809 00:35:16.032498 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:16.032499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:16.032508 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:16.032578 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:16.033467 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.034127 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.035258 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.035565 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.035659 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.038588 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:16.038589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:16.038590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:16.038596 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:16.040346 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.040727 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.040729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.040729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.040790 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.040985 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.040986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.040987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.041140 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:16.041306 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:16.041395 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 00:35:16.041482 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260809 00:35:16.042373 2635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff: No bootstrap required, opened a new log
I20260809 00:35:16.044286 2635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } }
Aug 09 00:35:16 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
I20260809 00:35:16.044441 2635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:16.044510 2635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c820a46c0bd4b1981a2c3148d8ad5ff, State: Initialized, Role: FOLLOWER
I20260809 00:35:16.044669 2635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } }
I20260809 00:35:16.045020 2635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [sys.catalog]: configured and running, proceeding with master startup.
W20260809 00:35:16.047274 2803 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:35:16.047389 2803 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 00:35:16 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:16.048434 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.040327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:35089 (local address 127.0.0.1:34232)
0809 00:35:16.040384 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:16.040397 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:16.040442 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:16.040582 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:16.040583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:16.040750 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:16.040785 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:16.040787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.041706 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.041707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:16.042472 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:16.042475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.043034 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.043035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:16.043099 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.043500 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:16.043505 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:16.044772 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:16.047340 (+ 2568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:16.047341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:16.047341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:16.047555 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:16.047788 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:16.047789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:16.047790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:16.047848 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:16.048137 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:16.048138 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:16.048180 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 00:35:16.048242 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:35:16.048335 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:35:16.048933 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.040434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34232 (local address 127.21.41.253:35089)
0809 00:35:16.040475 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:16.040477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:16.040484 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:35:16.040508 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:16.040509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:16.040520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:16.040575 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:16.040910 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.041618 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.042565 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.042908 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.042953 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.044864 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:16.044865 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:16.044866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:16.044872 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:16.047242 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.047643 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.047644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.047645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.047702 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.047916 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.047916 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.047917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.048051 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:16.048110 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:16.048744 (+ 634us) server_negotiation.cc:299] Negotiation successful
0809 00:35:16.048830 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:35:16.049274 2785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:35:16.048108 2567 master_runner.cc:186] Error getting master registration for 127.21.41.253:35089: OK, instance_id { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" instance_seqno: 1786235715851322 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:35:16.051021 2567 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:35:16.051221 2792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } } }
I20260809 00:35:16.051420 2792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:35:16.051647 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.042284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.21.41.254:41985)
0809 00:35:16.042320 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:16.042322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:16.042330 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:35:16.042376 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:16.042376 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:16.042386 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:16.042451 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:16.043035 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.043770 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.044776 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.045073 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.045149 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.048239 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:16.048240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:16.048241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:16.048248 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:16.049988 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.050462 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.050463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.050463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.050524 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.050730 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.050731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.050731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.050859 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:16.050992 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:16.051481 (+ 489us) server_negotiation.cc:299] Negotiation successful
0809 00:35:16.051567 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 09 00:35:16 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
I20260809 00:35:16.057040 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.049534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43972)
0809 00:35:16.049728 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:16.049742 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:16.049791 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:16.049929 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:16.049929 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:16.050071 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:16.050097 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:16.050098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.050954 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.050955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:16.051686 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:16.051689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.052294 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.052294 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:16.052340 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.052729 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:16.052734 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:16.053992 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:16.055818 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:16.055819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:16.055819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:16.056036 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:16.056277 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:16.056278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:16.056278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:16.056344 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:16.056722 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:16.056723 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:16.056768 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 00:35:16.056848 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 00:35:16.056947 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260809 00:35:16.057150 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.049563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.21.41.252:38335)
0809 00:35:16.049594 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:16.049595 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:16.049836 (+ 241us) server_negotiation.cc:407] Connection header received
0809 00:35:16.049852 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:16.049852 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:16.049862 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:16.049900 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:16.050162 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.050867 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.051892 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.052185 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.052246 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.054069 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:16.054070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:16.054070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:16.054076 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:16.055730 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.056133 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.056134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.056135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.056193 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.056422 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.056423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.056423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.056634 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:16.056706 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:16.056896 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 00:35:16.057022 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":16}
Aug 09 00:35:16 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
I20260809 00:35:16.059091 2785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9: Bootstrap starting.
I20260809 00:35:16.059846 2785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:16.060150 2785 log.cc:824] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:16.060734 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.052503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34238 (local address 127.21.41.253:35089)
0809 00:35:16.052540 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:16.052542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:16.052551 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:35:16.052626 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:16.052627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:16.052634 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:16.052668 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:16.053238 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.053954 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.054986 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.055270 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.055340 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.058003 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:16.058004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:16.058005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:16.058012 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:16.059558 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.059928 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.059929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.059929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.059990 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.060201 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.060202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.060203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.060310 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:16.060503 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:16.060539 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 00:35:16.060618 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:35:16.060968 2785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9: No bootstrap required, opened a new log
I20260809 00:35:16.062207 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:35:16.062997 2785 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } }
I20260809 00:35:16.063141 2785 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:16.063174 2785 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7cfe89d74a1476f96f16b87958653c9, State: Initialized, Role: FOLLOWER
I20260809 00:35:16.063289 2785 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } }
I20260809 00:35:16.063583 2785 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:35:16.063911 2810 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } } }
I20260809 00:35:16.064126 2810 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:35:16.066824 2717 master_runner.cc:186] Error getting master registration for 127.21.41.253:35089: OK, instance_id { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" instance_seqno: 1786235715851322 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:35:16.069621 2717 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 00:35:16.071470 2824 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:35:16.071846 2824 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:35:16.073796 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.064019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47972 (local address 127.21.41.254:41985)
0809 00:35:16.064063 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:16.064065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:16.064311 (+ 246us) server_negotiation.cc:407] Connection header received
0809 00:35:16.064386 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:16.064386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:16.064400 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:16.064453 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:16.065635 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.066468 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.067491 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.067810 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.067858 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.069458 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:16.069460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:16.069460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:16.069467 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:16.071657 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.072414 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.072415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.072416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.072474 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.073128 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.073130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.073130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.073283 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:16.073389 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:16.073619 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 00:35:16.073700 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:35:16.075903 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.064962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43980 (local address 127.21.41.252:38335)
0809 00:35:16.065003 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:16.065005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:16.065013 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:35:16.065033 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:16.065033 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:16.065043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:16.065097 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:16.068132 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.068893 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.069970 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.070286 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.070394 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.071755 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:16.071756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:16.071757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:16.071763 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:16.073499 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.075055 (+ 1556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.075056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.075057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.075112 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.075312 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.075312 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.075313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.075428 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:16.075504 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:16.075734 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 00:35:16.075813 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:35:16.076216 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.064059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34254 (local address 127.21.41.253:35089)
0809 00:35:16.064093 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:16.064095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:16.064652 (+ 557us) server_negotiation.cc:407] Connection header received
0809 00:35:16.064795 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:16.064796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:16.064806 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:16.064838 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:16.065648 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.066443 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.070004 (+ 3561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.070366 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.070415 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.072642 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:16.072643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:16.072644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:16.072651 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:16.074789 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.075315 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.075317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.075317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.075377 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.075575 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.075576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.075577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.075721 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:16.075798 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:16.076039 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 00:35:16.076115 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 00:35:16.086138 2710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } attempt: 3
Aug 09 00:35:16 dist-test-slave-zpfg krb5kdc[2554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235715, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
I20260809 00:35:16.093107 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.086475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:38335 (local address 127.0.0.1:43982)
0809 00:35:16.086535 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:16.086549 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:16.086600 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:16.086728 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:16.086728 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:16.086880 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:16.086919 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:16.086920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.087554 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.087555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:16.088072 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:16.088073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.088439 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.088441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:16.088498 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.088936 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:16.088941 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:16.090109 (+ 1168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:16.092025 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:16.092027 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:16.092027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:16.092237 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:16.092433 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:16.092434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:16.092434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:16.092500 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:16.092780 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:16.092781 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:16.092842 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 00:35:16.092911 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 00:35:16.093011 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 00:35:16.093106 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:16.086512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43982 (local address 127.21.41.252:38335)
0809 00:35:16.086560 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:16.086562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:16.086587 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:35:16.086651 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:16.086652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:16.086662 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:16.086717 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:16.087003 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.087461 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.088162 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:16.088356 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:16.088430 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:16.090191 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:16.090193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:16.090193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:16.090199 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:16.091940 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.092309 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.092310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.092310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.092354 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:16.092589 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:16.092590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:16.092590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:16.092702 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:16.092788 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:16.092941 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 00:35:16.093012 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:35:16.094553 2710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6: Bootstrap starting.
I20260809 00:35:16.095224 2710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:16.095551 2710 log.cc:824] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:16.096376 2710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6: No bootstrap required, opened a new log
I20260809 00:35:16.097826 2710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } }
I20260809 00:35:16.097968 2710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:16.098047 2710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb2781858c441dabf6730d3a62319b6, State: Initialized, Role: FOLLOWER
I20260809 00:35:16.098207 2710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } }
I20260809 00:35:16.098548 2827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } } }
I20260809 00:35:16.098623 2710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:35:16.098690 2827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:35:16.098924 2663 tablet.cc:2369] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:35:16.101763 2838 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:35:16.101897 2838 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:35:16.102698 2642 master_runner.cc:263] No leader master found from master 3cb2781858c441dabf6730d3a62319b6
I20260809 00:35:16.102747 2642 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 00:35:16.181087 2588 tablet.cc:2369] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:35:16.211279 2810 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:35:16.211421 2810 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } }
I20260809 00:35:16.211656 2810 leader_election.cc:288] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c820a46c0bd4b1981a2c3148d8ad5ff (127.21.41.254:41985), 3cb2781858c441dabf6730d3a62319b6 (127.21.41.253:35089)
I20260809 00:35:16.211973 2684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7cfe89d74a1476f96f16b87958653c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cb2781858c441dabf6730d3a62319b6" is_pre_election: true
I20260809 00:35:16.212152 2684 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7cfe89d74a1476f96f16b87958653c9 in term 0.
I20260809 00:35:16.212585 2609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7cfe89d74a1476f96f16b87958653c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" is_pre_election: true
I20260809 00:35:16.212740 2609 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7cfe89d74a1476f96f16b87958653c9 in term 0.
I20260809 00:35:16.213035 2734 leader_election.cc:302] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c820a46c0bd4b1981a2c3148d8ad5ff, a7cfe89d74a1476f96f16b87958653c9; no voters:
I20260809 00:35:16.213261 2810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:35:16.213321 2810 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:35:16.213346 2810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:16.214282 2810 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } }
I20260809 00:35:16.214526 2810 leader_election.cc:288] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [CANDIDATE]: Term 1 election: Requested vote from peers 1c820a46c0bd4b1981a2c3148d8ad5ff (127.21.41.254:41985), 3cb2781858c441dabf6730d3a62319b6 (127.21.41.253:35089)
I20260809 00:35:16.214769 2684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7cfe89d74a1476f96f16b87958653c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cb2781858c441dabf6730d3a62319b6"
I20260809 00:35:16.214841 2684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:16.215728 2684 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7cfe89d74a1476f96f16b87958653c9 in term 1.
I20260809 00:35:16.216022 2609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7cfe89d74a1476f96f16b87958653c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff"
I20260809 00:35:16.216096 2609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:16.216917 2609 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7cfe89d74a1476f96f16b87958653c9 in term 1.
I20260809 00:35:16.217131 2734 leader_election.cc:302] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c820a46c0bd4b1981a2c3148d8ad5ff, a7cfe89d74a1476f96f16b87958653c9; no voters:
I20260809 00:35:16.217401 2810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:35:16.217535 2810 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [term 1 LEADER]: Becoming Leader. State: Replica: a7cfe89d74a1476f96f16b87958653c9, State: Running, Role: LEADER
I20260809 00:35:16.217613 2810 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } }
I20260809 00:35:16.217975 2810 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7cfe89d74a1476f96f16b87958653c9. Latest consensus state: current_term: 1 leader_uuid: "a7cfe89d74a1476f96f16b87958653c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } } }
I20260809 00:35:16.218031 2810 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:16.218531 2842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:35:16.218765 2842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:35:16.220676 2684 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [term 1 FOLLOWER]: Refusing update from remote peer a7cfe89d74a1476f96f16b87958653c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:16.221027 2810 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:16.222231 2827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7cfe89d74a1476f96f16b87958653c9. Latest consensus state: current_term: 1 leader_uuid: "a7cfe89d74a1476f96f16b87958653c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } } }
I20260809 00:35:16.222316 2827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:35:16.222972 2609 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [term 1 FOLLOWER]: Refusing update from remote peer a7cfe89d74a1476f96f16b87958653c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:16.223369 2810 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:16.224509 2792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7cfe89d74a1476f96f16b87958653c9. Latest consensus state: current_term: 1 leader_uuid: "a7cfe89d74a1476f96f16b87958653c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } } }
I20260809 00:35:16.224591 2792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:35:16.225571 2840 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7cfe89d74a1476f96f16b87958653c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } } }
I20260809 00:35:16.225647 2840 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:16.226145 2827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7cfe89d74a1476f96f16b87958653c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } } }
I20260809 00:35:16.226225 2827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:35:16.226493 2843 mvcc.cc:205] Tried to move back new op lower bound from 7316421493637902336 to 7316421493627990016. Current Snapshot: MvccSnapshot[applied={T|T < 7316421493637902336}]
I20260809 00:35:16.228029 2844 mvcc.cc:205] Tried to move back new op lower bound from 7316421493637902336 to 7316421493627990016. Current Snapshot: MvccSnapshot[applied={T|T < 7316421493637902336}]
I20260809 00:35:16.228602 2842 catalog_manager.cc:1347] Generated new cluster ID: 6d4e9eca4cfb4b99a1fde8ed1defeb88
I20260809 00:35:16.228649 2842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:35:16.235330 2810 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7cfe89d74a1476f96f16b87958653c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } } }
I20260809 00:35:16.235432 2810 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:16.235884 2792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7cfe89d74a1476f96f16b87958653c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c820a46c0bd4b1981a2c3148d8ad5ff" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41985 } } peers { permanent_uuid: "3cb2781858c441dabf6730d3a62319b6" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 35089 } } peers { permanent_uuid: "a7cfe89d74a1476f96f16b87958653c9" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 38335 } } }
I20260809 00:35:16.235983 2792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:35:16.236331 2846 mvcc.cc:205] Tried to move back new op lower bound from 7316421493637902336 to 7316421493627990016. Current Snapshot: MvccSnapshot[applied={T|T < 7316421493637902336}]
I20260809 00:35:16.244508 2842 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:35:16.245330 2842 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:35:16.252436 2842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7cfe89d74a1476f96f16b87958653c9: Generated new TSK 0
I20260809 00:35:16.252637 2842 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:35:17.047923 2803 catalog_manager.cc:1259] Loaded cluster ID: 6d4e9eca4cfb4b99a1fde8ed1defeb88
I20260809 00:35:17.048039 2803 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff: loading cluster ID for follower catalog manager: success
I20260809 00:35:17.048874 2803 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff: acquiring CA information for follower catalog manager: success
I20260809 00:35:17.049208 2803 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1c820a46c0bd4b1981a2c3148d8ad5ff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:35:17.102421 2838 catalog_manager.cc:1259] Loaded cluster ID: 6d4e9eca4cfb4b99a1fde8ed1defeb88
I20260809 00:35:17.102543 2838 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6: loading cluster ID for follower catalog manager: success
I20260809 00:35:17.103358 2838 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6: acquiring CA information for follower catalog manager: success
I20260809 00:35:17.103662 2838 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3cb2781858c441dabf6730d3a62319b6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:35:17.306944 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 2567
I20260809 00:35:17.312511 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 2642
I20260809 00:35:17.317569 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 2717
2026-08-09T00:35:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2924 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:35:17 dist-test-slave-zpfg krb5kdc[2857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:35:17 dist-test-slave-zpfg krb5kdc[2857](info): set up 2 sockets
Aug 09 00:35:17 dist-test-slave-zpfg krb5kdc[2857](info): commencing operation
krb5kdc: starting...
W20260809 00:35:18.360060 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:35:18 dist-test-slave-zpfg krb5kdc[2857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:35:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:35:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:35:18.485230 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:41045
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:41045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:18.569144 2870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:18.569375 2870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:18.569398 2870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:18.570678 2870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:35:18.570726 2870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:18.570741 2870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:18.570755 2870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:35:18.570766 2870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:35:18.572351 2870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:41045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:41045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:18.572654 2870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:18.572911 2870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:18.577863 2878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:18.577940 2875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:18.578044 2870 server_base.cc:1034] running on GCE node
W20260809 00:35:18.578161 2876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:18.578531 2870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:18.578850 2870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:18.580021 2870 hybrid_clock.cc:648] HybridClock initialized: now 1786235718579998 us; error 34 us; skew 500 ppm
Aug 09 00:35:18 dist-test-slave-zpfg krb5kdc[2857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235718, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:18.582592 2870 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:35:18.583079 2870 webserver.cc:460] Webserver started at http://127.21.41.254:45619/ using document root <none> and password file <none>
I20260809 00:35:18.583350 2870 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:18.583423 2870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:18.583571 2870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:18.584625 2870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "7f5010a021624b97be22b927e3a3c174"
format_stamp: "Formatted at 2026-08-09 00:35:18 on dist-test-slave-zpfg"
server_key: "33604894ce3c99b4821798a45f45759a"
server_key_iv: "c1ac237ca5a5883ba9f3406cc5c521bd"
server_key_version: "encryptionkey@0"
I20260809 00:35:18.584996 2870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "7f5010a021624b97be22b927e3a3c174"
format_stamp: "Formatted at 2026-08-09 00:35:18 on dist-test-slave-zpfg"
server_key: "33604894ce3c99b4821798a45f45759a"
server_key_iv: "c1ac237ca5a5883ba9f3406cc5c521bd"
server_key_version: "encryptionkey@0"
I20260809 00:35:18.586464 2870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:35:18.587210 2885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:18.587425 2870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:35:18.587543 2870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "7f5010a021624b97be22b927e3a3c174"
format_stamp: "Formatted at 2026-08-09 00:35:18 on dist-test-slave-zpfg"
server_key: "33604894ce3c99b4821798a45f45759a"
server_key_iv: "c1ac237ca5a5883ba9f3406cc5c521bd"
server_key_version: "encryptionkey@0"
I20260809 00:35:18.587620 2870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:18.605334 2870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:18.607319 2870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:18.607493 2870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:18.612618 2870 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:41045
I20260809 00:35:18.612640 2938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:41045 every 8 connection(s)
I20260809 00:35:18.613032 2870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:35:18.613883 2939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:18.616446 2939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174: Bootstrap starting.
I20260809 00:35:18.617053 2939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:18.617355 2939 log.cc:824] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:18.618037 2939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174: No bootstrap required, opened a new log
I20260809 00:35:18.619694 2939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5010a021624b97be22b927e3a3c174" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41045 } }
I20260809 00:35:18.619860 2939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:18.619922 2939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f5010a021624b97be22b927e3a3c174, State: Initialized, Role: FOLLOWER
I20260809 00:35:18.620088 2939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5010a021624b97be22b927e3a3c174" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41045 } }
I20260809 00:35:18.620165 2939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:35:18.620210 2939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:35:18.620266 2939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:18.620481 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 2870
I20260809 00:35:18.620594 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:35:18.620819 21671 external_mini_cluster.cc:1428] Setting key 194a62bee416b39ea83db28e756f5fb0
I20260809 00:35:18.622134 2939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5010a021624b97be22b927e3a3c174" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41045 } }
I20260809 00:35:18.622247 2939 leader_election.cc:302] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f5010a021624b97be22b927e3a3c174; no voters:
I20260809 00:35:18.622427 2939 leader_election.cc:288] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:35:18.622448 2944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:35:18.622566 2944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [term 1 LEADER]: Becoming Leader. State: Replica: 7f5010a021624b97be22b927e3a3c174, State: Running, Role: LEADER
I20260809 00:35:18.622651 2944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5010a021624b97be22b927e3a3c174" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41045 } }
I20260809 00:35:18.622694 2939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:35:18.622974 2945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f5010a021624b97be22b927e3a3c174" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5010a021624b97be22b927e3a3c174" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41045 } } }
I20260809 00:35:18.623088 2945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:18.623245 2946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f5010a021624b97be22b927e3a3c174. Latest consensus state: current_term: 1 leader_uuid: "7f5010a021624b97be22b927e3a3c174" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5010a021624b97be22b927e3a3c174" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41045 } } }
I20260809 00:35:18.623320 2946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174 [sys.catalog]: This master's current role is: LEADER
W20260809 00:35:18.624447 2959 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:35:18.624547 2959 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:35:18.624897 2950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:35:18.625078 2950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:35:18.626753 2950 catalog_manager.cc:1347] Generated new cluster ID: babc60d5715f454796ee1b0bb4dac050
I20260809 00:35:18.626821 2950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:35:18 dist-test-slave-zpfg krb5kdc[2857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:18.632886 2950 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:35:18.633338 2943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:18.621354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55060 (local address 127.21.41.254:41045)
0809 00:35:18.621605 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:18.621617 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:18.621664 (+ 47us) server_negotiation.cc:407] Connection header received
0809 00:35:18.621832 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:18.621840 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:18.621900 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:18.622051 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:18.622887 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:18.623999 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:18.625005 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:18.625620 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:18.625735 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:18.628634 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:18.628638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:18.628639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:18.628650 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:18.631404 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:18.631830 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:18.631833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:18.631833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:18.631952 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:18.632385 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:18.632386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:18.632386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:18.632612 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:18.632731 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:18.632946 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 00:35:18.633099 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":134,"threads_started":1}
I20260809 00:35:18.633371 2950 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:35:18.634603 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:35:18.644726 2950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f5010a021624b97be22b927e3a3c174: Generated new TSK 0
I20260809 00:35:18.644927 2950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:35:18.688982 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 2870
2026-08-09T00:35:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1370 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:35:18 dist-test-slave-zpfg krb5kdc[2966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:35:18 dist-test-slave-zpfg krb5kdc[2966](info): set up 2 sockets
Aug 09 00:35:18 dist-test-slave-zpfg krb5kdc[2966](info): commencing operation
krb5kdc: starting...
W20260809 00:35:19.752117 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:35:19 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:35:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:35:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:35:19.876894 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45035
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:45035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:19.958603 2979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:19.958923 2979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:19.958961 2979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:19.960273 2979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:35:19.960318 2979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:19.960333 2979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:19.960346 2979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:35:19.960358 2979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:35:19.961907 2979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:45035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:19.962105 2979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:19.962277 2979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:19.967139 2984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:19.967106 2987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:19.967110 2985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:19.967366 2979 server_base.cc:1034] running on GCE node
I20260809 00:35:19.967739 2979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:19.968043 2979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:19.969213 2979 hybrid_clock.cc:648] HybridClock initialized: now 1786235719969181 us; error 44 us; skew 500 ppm
Aug 09 00:35:19 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235719, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:19.972064 2979 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:35:19.972512 2979 webserver.cc:460] Webserver started at http://127.21.41.254:38101/ using document root <none> and password file <none>
I20260809 00:35:19.972715 2979 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:19.972764 2979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:19.972852 2979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:19.973870 2979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "55934827fc854cbda74871c44d5de557"
format_stamp: "Formatted at 2026-08-09 00:35:19 on dist-test-slave-zpfg"
server_key: "39519b2148b45c08e925e4e2714bc535"
server_key_iv: "9a3ccf3d4b908c4b77d4b23099aa9697"
server_key_version: "encryptionkey@0"
I20260809 00:35:19.974174 2979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "55934827fc854cbda74871c44d5de557"
format_stamp: "Formatted at 2026-08-09 00:35:19 on dist-test-slave-zpfg"
server_key: "39519b2148b45c08e925e4e2714bc535"
server_key_iv: "9a3ccf3d4b908c4b77d4b23099aa9697"
server_key_version: "encryptionkey@0"
I20260809 00:35:19.975502 2979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:35:19.976325 2994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:19.976518 2979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:19.976608 2979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "55934827fc854cbda74871c44d5de557"
format_stamp: "Formatted at 2026-08-09 00:35:19 on dist-test-slave-zpfg"
server_key: "39519b2148b45c08e925e4e2714bc535"
server_key_iv: "9a3ccf3d4b908c4b77d4b23099aa9697"
server_key_version: "encryptionkey@0"
I20260809 00:35:19.976662 2979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:19.999398 2979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:20.001802 2979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:20.001977 2979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:20.006129 2979 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:45035
I20260809 00:35:20.006186 3047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:45035 every 8 connection(s)
I20260809 00:35:20.006620 2979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:35:20.007643 3048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:20.010067 3048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Bootstrap starting.
I20260809 00:35:20.010685 3048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:20.010982 3048 log.cc:824] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:20.011731 3048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: No bootstrap required, opened a new log
I20260809 00:35:20.012202 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 2979
I20260809 00:35:20.012291 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:35:20.012522 21671 external_mini_cluster.cc:1428] Setting key 137bb10b629e7622c30fcec85b61ef1f
I20260809 00:35:20.014020 3048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:20.014201 3048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:20.014240 3048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55934827fc854cbda74871c44d5de557, State: Initialized, Role: FOLLOWER
I20260809 00:35:20.014427 3048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:20.014497 3048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:35:20.014519 3048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:35:20.014585 3048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:20.015697 3048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:20.015825 3048 leader_election.cc:302] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55934827fc854cbda74871c44d5de557; no voters:
I20260809 00:35:20.015993 3048 leader_election.cc:288] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:35:20.016054 3053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:35:20.016232 3048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:35:20.016230 3053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 1 LEADER]: Becoming Leader. State: Replica: 55934827fc854cbda74871c44d5de557, State: Running, Role: LEADER
I20260809 00:35:20.016544 3053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:20.016924 3055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55934827fc854cbda74871c44d5de557. Latest consensus state: current_term: 1 leader_uuid: "55934827fc854cbda74871c44d5de557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } } }
I20260809 00:35:20.017053 3055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:20.017491 3054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55934827fc854cbda74871c44d5de557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } } }
I20260809 00:35:20.017572 3054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:20.017799 3062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:35:20.018577 3062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 00:35:20 dist-test-slave-zpfg krb5kdc[2966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:20.022887 3062 catalog_manager.cc:1347] Generated new cluster ID: ad8ea6ed956d428f9cbe2a22c575c996
I20260809 00:35:20.022955 3062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:35:20.025664 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.013063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.21.41.254:45035)
0809 00:35:20.013291 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:20.013301 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:20.013345 (+ 44us) server_negotiation.cc:407] Connection header received
0809 00:35:20.013979 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:20.013988 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:20.014057 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:20.014212 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:20.015054 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.016498 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.017594 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.018243 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.018331 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.021193 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:20.021197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:20.021197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:20.021209 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:20.023748 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:20.024200 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:20.024204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:20.024204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:20.024364 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:20.024583 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:20.024583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:20.024584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:20.024937 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:20.025110 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:20.025188 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 00:35:20.025375 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":100,"threads_started":1}
I20260809 00:35:20.038033 3062 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:35:20.038813 3062 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:35:20.049578 3062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Generated new TSK 0
I20260809 00:35:20.049836 3062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:35:20.086290 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45035
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:20.172322 3074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:20.172549 3074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:20.172577 3074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:20.173883 3074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:20.173928 3074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:20.173957 3074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:35:20.175556 3074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:20.176121 3074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:20.176405 3074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:20.181710 3079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:20.181725 3080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:20.181988 3074 server_base.cc:1034] running on GCE node
W20260809 00:35:20.181737 3082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:20.182286 3074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:20.182569 3074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:20.183709 3074 hybrid_clock.cc:648] HybridClock initialized: now 1786235720183696 us; error 28 us; skew 500 ppm
Aug 09 00:35:20 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235720, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:20.186160 3074 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:35:20.186645 3074 webserver.cc:460] Webserver started at http://127.21.41.193:33379/ using document root <none> and password file <none>
I20260809 00:35:20.186864 3074 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:20.186906 3074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:20.186993 3074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:20.188016 3074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "55efadde246743129db9b2632ce02bac"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "bf30b3e8c2a765801bad1238c9982cd2"
server_key_iv: "3c4dd410b2c69f18e176aefdc705d111"
server_key_version: "encryptionkey@0"
I20260809 00:35:20.188308 3074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "55efadde246743129db9b2632ce02bac"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "bf30b3e8c2a765801bad1238c9982cd2"
server_key_iv: "3c4dd410b2c69f18e176aefdc705d111"
server_key_version: "encryptionkey@0"
I20260809 00:35:20.189558 3074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:35:20.190255 3089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:20.190438 3074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:35:20.190526 3074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "55efadde246743129db9b2632ce02bac"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "bf30b3e8c2a765801bad1238c9982cd2"
server_key_iv: "3c4dd410b2c69f18e176aefdc705d111"
server_key_version: "encryptionkey@0"
I20260809 00:35:20.190582 3074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:20.212538 3074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:20.214560 3074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:20.214680 3074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:20.214890 3074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:20.215229 3074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:20.215286 3074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:20.215323 3074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:20.215371 3074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:20.221396 3074 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:34601
I20260809 00:35:20.221480 3204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:34601 every 8 connection(s)
I20260809 00:35:20.221812 3074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:35:20 dist-test-slave-zpfg krb5kdc[2966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235720, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:20.230815 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.222612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:56615 (local address 127.21.41.254:45035)
0809 00:35:20.222687 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:20.222688 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:20.222911 (+ 223us) server_negotiation.cc:407] Connection header received
0809 00:35:20.223387 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:20.223388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:20.223402 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:20.223462 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:20.224375 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.224877 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.225847 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.226052 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.226105 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.228310 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:20.228312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:20.228312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:20.228318 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:20.229560 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:20.229936 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:20.229936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:20.229937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:20.229988 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:20.230197 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:20.230197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:20.230198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:20.230322 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:20.230444 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:20.230673 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 00:35:20.230746 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:35:20.231034 3207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.222572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:56615)
0809 00:35:20.222748 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:20.222791 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:20.223297 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:20.223518 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:20.223521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:20.224098 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:20.224291 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.224296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.224960 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.224961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.225764 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.225766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.226128 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.226129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.226233 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.226829 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:20.226838 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:20.228209 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:20.229636 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:20.229642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:20.229644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:20.229869 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:20.230047 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:20.230048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:20.230048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:20.230144 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:20.230437 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:20.230442 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:20.230558 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 00:35:20.230645 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 00:35:20.230768 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260809 00:35:20.231678 3205 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45035
I20260809 00:35:20.231786 3205 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:20.232009 3205 heartbeater.cc:511] Master 127.21.41.254:45035 requested a full tablet report, sending...
I20260809 00:35:20.232041 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 3074
I20260809 00:35:20.232142 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:35:20.232419 21671 external_mini_cluster.cc:1428] Setting key 951a99c2e88d4faa31873812e3b206f8
I20260809 00:35:20.232679 3011 ts_manager.cc:194] Registered new tserver with Master: 55efadde246743129db9b2632ce02bac (127.21.41.193:34601)
I20260809 00:35:20.233541 3011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:56615
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:35:20.258001 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45035
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:20.340421 3210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:20.340657 3210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:20.340694 3210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:20.341846 3210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:20.341913 3210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:20.341955 3210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:35:20.343422 3210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:20.343691 3210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:20.343931 3210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:20.348814 3218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:20.348819 3216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:20.348835 3215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:20.349103 3210 server_base.cc:1034] running on GCE node
I20260809 00:35:20.349298 3210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:20.349509 3210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:20.350641 3210 hybrid_clock.cc:648] HybridClock initialized: now 1786235720350628 us; error 31 us; skew 500 ppm
Aug 09 00:35:20 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235720, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:20.353120 3210 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:35:20.353613 3210 webserver.cc:460] Webserver started at http://127.21.41.194:44871/ using document root <none> and password file <none>
I20260809 00:35:20.353837 3210 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:20.353881 3210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:20.354027 3210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:20.355175 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "ab7ecb09d7c24b3caddde3419f7455bd"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "da23c5cb50295a0a184bee3abb2c90ce"
server_key_iv: "d5ffb61f26138d0193ac343973c0b2ae"
server_key_version: "encryptionkey@0"
I20260809 00:35:20.355540 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "ab7ecb09d7c24b3caddde3419f7455bd"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "da23c5cb50295a0a184bee3abb2c90ce"
server_key_iv: "d5ffb61f26138d0193ac343973c0b2ae"
server_key_version: "encryptionkey@0"
I20260809 00:35:20.356873 3210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:35:20.357604 3225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:20.357779 3210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:35:20.357867 3210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "ab7ecb09d7c24b3caddde3419f7455bd"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "da23c5cb50295a0a184bee3abb2c90ce"
server_key_iv: "d5ffb61f26138d0193ac343973c0b2ae"
server_key_version: "encryptionkey@0"
I20260809 00:35:20.357942 3210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:20.369966 3210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:20.371959 3210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:20.372102 3210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:20.372367 3210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:20.372680 3210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:20.372712 3210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:20.372735 3210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:20.372747 3210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:20.378860 3210 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:40541
I20260809 00:35:20.378916 3339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:40541 every 8 connection(s)
I20260809 00:35:20.379243 3210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:35:20.382886 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 3210
I20260809 00:35:20.382988 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:35:20.383240 21671 external_mini_cluster.cc:1428] Setting key f009efe17a0370203261c4109106bae4
Aug 09 00:35:20 dist-test-slave-zpfg krb5kdc[2966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235720, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:20.389650 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.380025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:37163)
0809 00:35:20.380182 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:20.380218 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:20.380750 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:20.380991 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:20.380995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:20.381560 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:20.381754 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.381758 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.382495 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.382495 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.383458 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.383462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.383968 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.383969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.384117 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.384660 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:20.384668 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:20.386241 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:20.388253 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:20.388261 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:20.388263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:20.388491 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:20.388715 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:20.388715 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:20.388715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:20.388809 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:20.389125 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:20.389129 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:20.389248 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 00:35:20.389369 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 00:35:20.389486 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":105,"threads_started":1}
I20260809 00:35:20.389676 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.380006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:37163 (local address 127.21.41.254:45035)
0809 00:35:20.380072 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:20.380074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:20.380287 (+ 213us) server_negotiation.cc:407] Connection header received
0809 00:35:20.380846 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:20.380847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:20.380859 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:20.380912 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:20.381846 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.382413 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.383574 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.383876 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.383950 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.386382 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:20.386384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:20.386384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:20.386392 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:20.388155 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:20.388573 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:20.388574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:20.388575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:20.388641 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:20.388880 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:20.388881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:20.388882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:20.389047 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:20.389125 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:20.389425 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 00:35:20.389522 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:35:20.390316 3340 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45035
I20260809 00:35:20.390470 3340 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:20.390650 3340 heartbeater.cc:511] Master 127.21.41.254:45035 requested a full tablet report, sending...
I20260809 00:35:20.391079 3011 ts_manager.cc:194] Registered new tserver with Master: ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541)
I20260809 00:35:20.391454 3011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:37163
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:35:20.412799 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45035
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:20.499280 3345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:20.499528 3345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:20.499571 3345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:20.500718 3345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:20.500787 3345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:20.500859 3345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:35:20.502308 3345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:20.502633 3345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:20.502871 3345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:20.507932 3350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:20.507993 3345 server_base.cc:1034] running on GCE node
W20260809 00:35:20.507961 3353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:20.507959 3351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:20.508327 3345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:20.508591 3345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:20.509749 3345 hybrid_clock.cc:648] HybridClock initialized: now 1786235720509745 us; error 36 us; skew 500 ppm
Aug 09 00:35:20 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235720, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:20.512216 3345 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:35:20.512671 3345 webserver.cc:460] Webserver started at http://127.21.41.195:40209/ using document root <none> and password file <none>
I20260809 00:35:20.512910 3345 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:20.512979 3345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:20.513128 3345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:20.514129 3345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "e07435db7b0e4b55810de6a80af41504"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "7212e622a44e766d2d7e7bcf5bc04193"
server_key_iv: "1a1245af8612ddfb3c80926e766df019"
server_key_version: "encryptionkey@0"
I20260809 00:35:20.514515 3345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "e07435db7b0e4b55810de6a80af41504"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "7212e622a44e766d2d7e7bcf5bc04193"
server_key_iv: "1a1245af8612ddfb3c80926e766df019"
server_key_version: "encryptionkey@0"
I20260809 00:35:20.515807 3345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:35:20.516587 3360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:20.516755 3345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:35:20.516855 3345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "e07435db7b0e4b55810de6a80af41504"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "7212e622a44e766d2d7e7bcf5bc04193"
server_key_iv: "1a1245af8612ddfb3c80926e766df019"
server_key_version: "encryptionkey@0"
I20260809 00:35:20.516959 3345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:20.530805 3345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:20.533010 3345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:20.533176 3345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:20.533432 3345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:20.533766 3345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:20.533823 3345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:20.533867 3345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:20.533890 3345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:20.540117 3345 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:43563
I20260809 00:35:20.540164 3475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:43563 every 8 connection(s)
I20260809 00:35:20.540517 3345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:35:20 dist-test-slave-zpfg krb5kdc[2966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235720, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:20.548210 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 3345
I20260809 00:35:20.548355 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:35:20.548581 21671 external_mini_cluster.cc:1428] Setting key 5838cc088e645c47075451e571ea6bb9
I20260809 00:35:20.550434 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.541240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:39089 (local address 127.21.41.254:45035)
0809 00:35:20.541316 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:20.541318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:20.541561 (+ 243us) server_negotiation.cc:407] Connection header received
0809 00:35:20.542103 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:20.542104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:20.542116 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:20.542177 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:20.543256 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.543779 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.544802 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.545007 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.545081 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.547334 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:20.547337 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:20.547337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:20.547346 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:20.548869 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:20.549314 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:20.549315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:20.549316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:20.549384 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:20.549688 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:20.549690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:20.549690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:20.549825 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:20.549905 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:20.550208 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 00:35:20.550305 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:35:20.550681 3478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.541247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:39089)
0809 00:35:20.541434 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:20.541471 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:20.541992 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:20.542266 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:20.542285 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:20.542950 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:20.543151 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.543156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.543876 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.543877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.544689 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.544691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.545089 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.545090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.545222 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.545763 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:20.545773 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:20.547206 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:20.548963 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:20.548973 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:20.548977 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:20.549231 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:20.549475 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:20.549476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:20.549476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:20.549612 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:20.549896 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:20.549904 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:20.550072 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 00:35:20.550187 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 00:35:20.550394 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260809 00:35:20.551152 3476 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45035
I20260809 00:35:20.551270 3476 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:20.551467 3476 heartbeater.cc:511] Master 127.21.41.254:45035 requested a full tablet report, sending...
I20260809 00:35:20.551890 3011 ts_manager.cc:194] Registered new tserver with Master: e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563)
I20260809 00:35:20.552279 3011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:39089
I20260809 00:35:20.560570 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:35:20.568612 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.562059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.21.41.254:45035)
0809 00:35:20.562136 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:20.562139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:20.562254 (+ 115us) server_negotiation.cc:407] Connection header received
0809 00:35:20.562374 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:20.562375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:20.562385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:20.562438 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:20.563049 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.563547 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.564285 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.564495 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.564572 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.565860 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:20.565862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:20.565862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:20.565869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:20.567359 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:20.567758 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:20.567760 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:20.567760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:20.567820 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:20.568026 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:20.568027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:20.568028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:20.568196 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:20.568307 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:20.568410 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 00:35:20.568493 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:35:20.570259 3011 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:35:20.571923 3011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:35:20.578778 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.574696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:40541 (local address 127.0.0.1:36332)
0809 00:35:20.574818 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:20.574828 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:20.574880 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:20.575065 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:20.575069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:20.575071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:20.575154 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.575156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.576497 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.576498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.577539 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.577542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.578414 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.578415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.578496 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.578511 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:35:20.578632 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 00:35:20.578695 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":56,"thread_start_us":66,"threads_started":1}
I20260809 00:35:20.578799 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.574691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36332 (local address 127.21.41.194:40541)
0809 00:35:20.574882 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:20.574887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:20.574917 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:35:20.574960 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:20.574962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:20.574980 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:20.575052 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:20.575242 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.576397 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.577638 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.578304 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.578407 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.578467 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:20.578667 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 00:35:20.578717 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":80,"threads_started":1}
I20260809 00:35:20.579310 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.574508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59872 (local address 127.21.41.193:34601)
0809 00:35:20.575197 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:20.575202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:20.575211 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:35:20.575250 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:20.575251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:20.575266 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:20.575358 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:20.575461 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.576525 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.577884 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.578682 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.578747 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.578798 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:20.579170 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 00:35:20.579217 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":95,"threads_started":1}
I20260809 00:35:20.579586 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.574377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:43563 (local address 127.0.0.1:38614)
0809 00:35:20.574705 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:20.574751 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:20.574834 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:20.575243 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:20.575244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:20.575246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:20.575290 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.575291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.576374 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.576375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.577800 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.577802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.579340 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.579341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.579412 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.579417 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:35:20.579468 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:35:20.579509 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":68,"thread_start_us":74,"threads_started":1}
I20260809 00:35:20.579094 3491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.574378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:34601 (local address 127.0.0.1:59872)
0809 00:35:20.574811 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:20.574823 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:20.574880 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:20.575357 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:20.575358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:20.575360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:20.575394 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.575395 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.576634 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.576635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.577794 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.577797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.578847 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.578848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.578921 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.578925 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:35:20.579000 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 00:35:20.579046 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":30,"thread_start_us":80,"threads_started":1}
I20260809 00:35:20.579824 3490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.574617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38614 (local address 127.21.41.195:43563)
0809 00:35:20.575006 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:20.575011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:20.575021 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:35:20.575063 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:20.575065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:20.575081 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:20.575169 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:20.575401 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.576261 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.578411 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.579258 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.579337 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.579392 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:20.579696 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 00:35:20.579751 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":96,"threads_started":1}
I20260809 00:35:20.580109 3273 tablet_service.cc:1467] Processing CreateTablet for tablet 1842dbf5156b471789862fe01a9d865e (DEFAULT_TABLE table=test-table [id=23dfb0c3418b4d15a76108448f951b9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:20.580258 3138 tablet_service.cc:1467] Processing CreateTablet for tablet 1842dbf5156b471789862fe01a9d865e (DEFAULT_TABLE table=test-table [id=23dfb0c3418b4d15a76108448f951b9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:20.580507 3273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1842dbf5156b471789862fe01a9d865e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:20.580545 3138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1842dbf5156b471789862fe01a9d865e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:20.584039 3493 tablet_bootstrap.cc:492] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: Bootstrap starting.
I20260809 00:35:20.584868 3493 tablet_bootstrap.cc:654] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:20.584821 3408 tablet_service.cc:1467] Processing CreateTablet for tablet 1842dbf5156b471789862fe01a9d865e (DEFAULT_TABLE table=test-table [id=23dfb0c3418b4d15a76108448f951b9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:20.585165 3408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1842dbf5156b471789862fe01a9d865e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:20.585469 3493 log.cc:824] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:20.585858 3494 tablet_bootstrap.cc:492] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Bootstrap starting.
I20260809 00:35:20.586186 3493 tablet_bootstrap.cc:492] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: No bootstrap required, opened a new log
I20260809 00:35:20.586275 3493 ts_tablet_manager.cc:1397] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:35:20.586577 3494 tablet_bootstrap.cc:654] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:20.587189 3494 log.cc:824] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:20.588081 3494 tablet_bootstrap.cc:492] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: No bootstrap required, opened a new log
I20260809 00:35:20.588156 3494 ts_tablet_manager.cc:1397] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:35:20.588198 3493 raft_consensus.cc:348] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } }
I20260809 00:35:20.588371 3493 raft_consensus.cc:374] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:20.588442 3493 raft_consensus.cc:729] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab7ecb09d7c24b3caddde3419f7455bd, State: Initialized, Role: FOLLOWER
I20260809 00:35:20.588590 3493 consensus_queue.cc:260] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } }
I20260809 00:35:20.589015 3493 ts_tablet_manager.cc:1428] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 00:35:20.589051 3340 heartbeater.cc:503] Master 127.21.41.254:45035 was elected leader, sending a full tablet report...
I20260809 00:35:20.590059 3494 raft_consensus.cc:348] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } }
I20260809 00:35:20.590214 3494 raft_consensus.cc:374] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:20.590268 3494 raft_consensus.cc:729] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55efadde246743129db9b2632ce02bac, State: Initialized, Role: FOLLOWER
I20260809 00:35:20.590430 3494 consensus_queue.cc:260] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } }
I20260809 00:35:20.590483 3497 tablet_bootstrap.cc:492] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504: Bootstrap starting.
I20260809 00:35:20.590736 3494 ts_tablet_manager.cc:1428] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:35:20.590770 3205 heartbeater.cc:503] Master 127.21.41.254:45035 was elected leader, sending a full tablet report...
I20260809 00:35:20.591156 3497 tablet_bootstrap.cc:654] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:20.591759 3497 log.cc:824] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:20.592480 3497 tablet_bootstrap.cc:492] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504: No bootstrap required, opened a new log
I20260809 00:35:20.592532 3497 ts_tablet_manager.cc:1397] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:35:20.593940 3497 raft_consensus.cc:348] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } }
I20260809 00:35:20.594056 3497 raft_consensus.cc:374] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:20.594077 3497 raft_consensus.cc:729] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07435db7b0e4b55810de6a80af41504, State: Initialized, Role: FOLLOWER
I20260809 00:35:20.594151 3497 consensus_queue.cc:260] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } }
I20260809 00:35:20.594448 3497 ts_tablet_manager.cc:1428] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:35:20.594485 3476 heartbeater.cc:503] Master 127.21.41.254:45035 was elected leader, sending a full tablet report...
W20260809 00:35:20.629853 3341 tablet.cc:2369] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:35:20.662741 3498 raft_consensus.cc:482] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:35:20.662897 3498 raft_consensus.cc:504] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } }
I20260809 00:35:20.663378 3498 leader_election.cc:288] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563), 55efadde246743129db9b2632ce02bac (127.21.41.193:34601)
I20260809 00:35:20.666590 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.663790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:48693 (local address 127.21.41.193:34601)
0809 00:35:20.663850 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:20.663853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:20.664094 (+ 241us) server_negotiation.cc:407] Connection header received
0809 00:35:20.664164 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:20.664165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:20.664177 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:20.664222 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:20.664387 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.664876 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.665807 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.666206 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.666294 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.666302 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:20.666483 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 00:35:20.666521 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:35:20.666601 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.663714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:34601 (local address 127.21.41.194:48693)
0809 00:35:20.664017 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:20.664027 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:20.664100 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:20.664223 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:20.664224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:20.664225 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:20.664295 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.664297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.664964 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.664965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.665712 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.665714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.666293 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.666294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.666400 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.666404 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:35:20.666479 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 00:35:20.666516 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":226,"threads_started":1}
I20260809 00:35:20.666682 3490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.663650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:59663 (local address 127.21.41.195:43563)
0809 00:35:20.663711 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:20.663713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:20.663724 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:35:20.663754 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:20.663755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:20.663764 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:20.663824 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:20.664104 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.664728 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.665826 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.666258 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.666377 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.666387 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:20.666553 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 00:35:20.666591 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:35:20.666944 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.663574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:43563 (local address 127.21.41.194:59663)
0809 00:35:20.663617 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:20.663631 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:20.663682 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:20.663889 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:20.663891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:20.663893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:20.663982 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.663985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.664832 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.664833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.665717 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:20.665719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.666377 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.666378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:20.666465 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.666468 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:35:20.666546 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:35:20.666588 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":13}
I20260809 00:35:20.667172 3159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1842dbf5156b471789862fe01a9d865e" candidate_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55efadde246743129db9b2632ce02bac" is_pre_election: true
I20260809 00:35:20.667366 3159 raft_consensus.cc:2454] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab7ecb09d7c24b3caddde3419f7455bd in term 0.
I20260809 00:35:20.667369 3430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1842dbf5156b471789862fe01a9d865e" candidate_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07435db7b0e4b55810de6a80af41504" is_pre_election: true
I20260809 00:35:20.667518 3430 raft_consensus.cc:2454] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab7ecb09d7c24b3caddde3419f7455bd in term 0.
I20260809 00:35:20.667613 3226 leader_election.cc:302] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55efadde246743129db9b2632ce02bac, ab7ecb09d7c24b3caddde3419f7455bd; no voters:
I20260809 00:35:20.667795 3498 raft_consensus.cc:2781] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:35:20.667860 3498 raft_consensus.cc:482] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:35:20.667893 3498 raft_consensus.cc:3037] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:20.668694 3498 raft_consensus.cc:504] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } }
I20260809 00:35:20.668867 3498 leader_election.cc:288] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [CANDIDATE]: Term 1 election: Requested vote from peers e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563), 55efadde246743129db9b2632ce02bac (127.21.41.193:34601)
I20260809 00:35:20.669029 3159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1842dbf5156b471789862fe01a9d865e" candidate_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55efadde246743129db9b2632ce02bac"
I20260809 00:35:20.669132 3159 raft_consensus.cc:3037] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:20.669128 3430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1842dbf5156b471789862fe01a9d865e" candidate_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07435db7b0e4b55810de6a80af41504"
I20260809 00:35:20.669282 3430 raft_consensus.cc:3037] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:20.669994 3159 raft_consensus.cc:2454] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab7ecb09d7c24b3caddde3419f7455bd in term 1.
I20260809 00:35:20.670100 3430 raft_consensus.cc:2454] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab7ecb09d7c24b3caddde3419f7455bd in term 1.
I20260809 00:35:20.670178 3226 leader_election.cc:302] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55efadde246743129db9b2632ce02bac, ab7ecb09d7c24b3caddde3419f7455bd; no voters:
I20260809 00:35:20.670289 3498 raft_consensus.cc:2781] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:35:20.670524 3498 raft_consensus.cc:686] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 1 LEADER]: Becoming Leader. State: Replica: ab7ecb09d7c24b3caddde3419f7455bd, State: Running, Role: LEADER
I20260809 00:35:20.670636 3498 consensus_queue.cc:237] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } }
I20260809 00:35:20.671531 3011 catalog_manager.cc:5495] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd reported cstate change: term changed from 0 to 1, leader changed from <none> to ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194). New cstate: current_term: 1 leader_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } health_report { overall_health: HEALTHY } } }
I20260809 00:35:20.695497 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.692884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36348 (local address 127.21.41.194:40541)
0809 00:35:20.692930 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:20.692946 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:20.692955 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:35:20.692996 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:20.692996 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:20.693039 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:20.693112 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:35:20.693277 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.693822 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.694673 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:20.694890 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:20.694967 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:20.695141 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:35:20.695252 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:35:20.695349 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:20.695369 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 00:35:20.695389 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:35:20.697454 3430 raft_consensus.cc:1261] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504 [term 1 FOLLOWER]: Refusing update from remote peer ab7ecb09d7c24b3caddde3419f7455bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:20.697456 3159 raft_consensus.cc:1261] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Refusing update from remote peer ab7ecb09d7c24b3caddde3419f7455bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:20.697816 3503 consensus_queue.cc:1035] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:20.697954 3498 consensus_queue.cc:1035] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:20.700443 3507 mvcc.cc:205] Tried to move back new op lower bound from 7316421511974150144 to 7316421511867551744. Current Snapshot: MvccSnapshot[applied={T|T < 7316421511974150144}]
I20260809 00:35:20.703449 3011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:35:20.703541 3011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:35:20.704319 3011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Sending DeleteTablet for 3 replicas of tablet 1842dbf5156b471789862fe01a9d865e
I20260809 00:35:20.704654 3408 tablet_service.cc:1514] Processing DeleteTablet for tablet 1842dbf5156b471789862fe01a9d865e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:20 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:38614
I20260809 00:35:20.704689 3273 tablet_service.cc:1514] Processing DeleteTablet for tablet 1842dbf5156b471789862fe01a9d865e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:20 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:36332
I20260809 00:35:20.704674 3138 tablet_service.cc:1514] Processing DeleteTablet for tablet 1842dbf5156b471789862fe01a9d865e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:20 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:59872
I20260809 00:35:20.704887 3518 tablet_replica.cc:330] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: stopping tablet replica
I20260809 00:35:20.704960 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 3074
I20260809 00:35:20.704975 3518 raft_consensus.cc:2229] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:35:20.705708 3518 raft_consensus.cc:2258] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:35:20.705878 3517 tablet_replica.cc:330] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504: stopping tablet replica
I20260809 00:35:20.705974 3517 raft_consensus.cc:2229] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:35:20.706071 3517 raft_consensus.cc:2258] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:35:20.706467 3518 ts_tablet_manager.cc:1905] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:35:20.706506 3517 ts_tablet_manager.cc:1905] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:35:20.708559 3517 ts_tablet_manager.cc:1918] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:35:20.708639 3517 log.cc:1196] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/1842dbf5156b471789862fe01a9d865e
I20260809 00:35:20.708757 3518 ts_tablet_manager.cc:1918] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:35:20.708889 3518 log.cc:1196] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/1842dbf5156b471789862fe01a9d865e
I20260809 00:35:20.708984 3517 ts_tablet_manager.cc:1939] T 1842dbf5156b471789862fe01a9d865e P e07435db7b0e4b55810de6a80af41504: Deleting consensus metadata
I20260809 00:35:20.709282 3518 ts_tablet_manager.cc:1939] T 1842dbf5156b471789862fe01a9d865e P ab7ecb09d7c24b3caddde3419f7455bd: Deleting consensus metadata
I20260809 00:35:20.709393 2996 catalog_manager.cc:4841] TS e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563): tablet 1842dbf5156b471789862fe01a9d865e (table test-table [id=23dfb0c3418b4d15a76108448f951b9f]) successfully deleted
I20260809 00:35:20.709765 2996 catalog_manager.cc:4841] TS ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541): tablet 1842dbf5156b471789862fe01a9d865e (table test-table [id=23dfb0c3418b4d15a76108448f951b9f]) successfully deleted
W20260809 00:35:20.711972 2995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:34601 (error 108)
I20260809 00:35:20.712153 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 3210
I20260809 00:35:20.713248 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:20.713106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:34601 (local address 127.0.0.1:59888)
0809 00:35:20.713165 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:20.713211 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 00:35:20.713449 2995 catalog_manager.cc:4584] TS 55efadde246743129db9b2632ce02bac (127.21.41.193:34601): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1842dbf5156b471789862fe01a9d865e: Network error: Client connection negotiation failed: client connection to 127.21.41.193:34601: connect: Connection refused (error 111)
I20260809 00:35:20.718087 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 3345
I20260809 00:35:20.723196 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 2979
I20260809 00:35:20.728825 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45035
--webserver_interface=127.21.41.254
--webserver_port=38101
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:45035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:20.813753 3521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:20.813958 3521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:20.813980 3521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:20.815630 3521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:35:20.815675 3521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:20.815689 3521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:20.815702 3521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:35:20.815716 3521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:35:20.817113 3521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:45035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=38101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:20.817309 3521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:20.817492 3521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:20.826377 3530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:20.826363 3527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:20.826385 3528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:20.826817 3521 server_base.cc:1034] running on GCE node
I20260809 00:35:20.827044 3521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:20.827323 3521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:20.828464 3521 hybrid_clock.cc:648] HybridClock initialized: now 1786235720828451 us; error 28 us; skew 500 ppm
Aug 09 00:35:20 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235720, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:20.835100 3521 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:35:20.835577 3521 webserver.cc:460] Webserver started at http://127.21.41.254:38101/ using document root <none> and password file <none>
I20260809 00:35:20.835799 3521 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:20.835855 3521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:20.837294 3521 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:20.838059 3537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:20.838238 3521 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:20.838361 3521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "55934827fc854cbda74871c44d5de557"
format_stamp: "Formatted at 2026-08-09 00:35:19 on dist-test-slave-zpfg"
server_key: "39519b2148b45c08e925e4e2714bc535"
server_key_iv: "9a3ccf3d4b908c4b77d4b23099aa9697"
server_key_version: "encryptionkey@0"
I20260809 00:35:20.838881 3521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:20.870821 3521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:20.873291 3521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:20.873463 3521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:20.877650 3521 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:45035
I20260809 00:35:20.877712 3590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:45035 every 8 connection(s)
I20260809 00:35:20.878129 3521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:35:20.879781 3591 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:35:20.880476 3591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Bootstrap starting.
I20260809 00:35:20.884581 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 3521
I20260809 00:35:20.884953 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:34601
--local_ip_for_outbound_sockets=127.21.41.193
--tserver_master_addrs=127.21.41.254:45035
--webserver_port=33379
--webserver_interface=127.21.41.193
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:35:20.889518 3591 log.cc:824] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:20.891636 3591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:35:20.891955 3591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Bootstrap complete.
I20260809 00:35:20.894266 3591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:20.894579 3591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55934827fc854cbda74871c44d5de557, State: Initialized, Role: FOLLOWER
I20260809 00:35:20.894771 3591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:20.894850 3591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:35:20.894907 3591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:35:20.894955 3591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:35:20.895980 3591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:20.896100 3591 leader_election.cc:302] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55934827fc854cbda74871c44d5de557; no voters:
I20260809 00:35:20.896319 3591 leader_election.cc:288] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:35:20.896358 3595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:35:20.896462 3595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 2 LEADER]: Becoming Leader. State: Replica: 55934827fc854cbda74871c44d5de557, State: Running, Role: LEADER
I20260809 00:35:20.896593 3591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:35:20.896610 3595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:20.897010 3596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "55934827fc854cbda74871c44d5de557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } } }
I20260809 00:35:20.897099 3596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:20.897482 3597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55934827fc854cbda74871c44d5de557. Latest consensus state: current_term: 2 leader_uuid: "55934827fc854cbda74871c44d5de557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } } }
I20260809 00:35:20.897554 3597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:20.897691 3602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:35:20.898671 3602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:35:20.899001 3602 catalog_manager.cc:1259] Loaded cluster ID: ad8ea6ed956d428f9cbe2a22c575c996
I20260809 00:35:20.899112 3602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:35:20.899358 3602 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 00:35:20.902029 3602 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 00:35:20.907394 3602 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:35:20.907618 3602 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Loaded TSK: 0
I20260809 00:35:20.907899 3602 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:35:20.981586 3593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:20.981801 3593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:20.981827 3593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:20.983116 3593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:20.983161 3593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:20.983191 3593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:35:20.984647 3593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:34601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=33379
--tserver_master_addrs=127.21.41.254:45035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:20.984889 3593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:20.985138 3593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:20.990141 3619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:20.990280 3621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:20.990417 3593 server_base.cc:1034] running on GCE node
W20260809 00:35:20.990150 3618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:20.990729 3593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:20.991015 3593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:20.992192 3593 hybrid_clock.cc:648] HybridClock initialized: now 1786235720992165 us; error 44 us; skew 500 ppm
Aug 09 00:35:20 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235720, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:20.994848 3593 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:35:20.995316 3593 webserver.cc:460] Webserver started at http://127.21.41.193:33379/ using document root <none> and password file <none>
I20260809 00:35:20.995566 3593 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:20.995656 3593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:20.997319 3593 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:35:20.998183 3628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:20.998412 3593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:35:20.998512 3593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "55efadde246743129db9b2632ce02bac"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "bf30b3e8c2a765801bad1238c9982cd2"
server_key_iv: "3c4dd410b2c69f18e176aefdc705d111"
server_key_version: "encryptionkey@0"
I20260809 00:35:20.998948 3593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:21.021098 3593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:21.023713 3593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:21.023874 3593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:21.024192 3593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:21.024727 3635 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:35:21.026247 3593 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:35:21.026311 3593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:35:21.026366 3593 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:35:21.027211 3593 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:35:21.027259 3593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:35:21.027407 3635 tablet_bootstrap.cc:492] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Bootstrap starting.
I20260809 00:35:21.034260 3593 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:34601
I20260809 00:35:21.034788 3593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:35:21.035058 3743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:34601 every 8 connection(s)
I20260809 00:35:21.038439 3635 log.cc:824] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:21.040694 3635 tablet_bootstrap.cc:492] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:35:21.041033 3635 tablet_bootstrap.cc:492] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Bootstrap complete.
I20260809 00:35:21.041211 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 3593
I20260809 00:35:21.041492 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:40541
--local_ip_for_outbound_sockets=127.21.41.194
--tserver_master_addrs=127.21.41.254:45035
--webserver_port=44871
--webserver_interface=127.21.41.194
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:35:21.042328 3635 ts_tablet_manager.cc:1397] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.014s
Aug 09 00:35:21 dist-test-slave-zpfg krb5kdc[2966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235720, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:21.044322 3635 raft_consensus.cc:348] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } }
I20260809 00:35:21.044665 3635 raft_consensus.cc:729] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55efadde246743129db9b2632ce02bac, State: Initialized, Role: FOLLOWER
I20260809 00:35:21.044837 3635 consensus_queue.cc:260] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } } peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } }
I20260809 00:35:21.045140 3635 ts_tablet_manager.cc:1428] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 00:35:21.049149 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.036023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:43301 (local address 127.21.41.254:45035)
0809 00:35:21.037006 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:21.037016 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:21.037058 (+ 42us) server_negotiation.cc:407] Connection header received
0809 00:35:21.037204 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:21.037211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:21.037279 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:21.037437 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:21.038285 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.039641 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.041146 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.041937 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.042624 (+ 687us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.045029 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:21.045033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:21.045034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:21.045046 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:21.047074 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:21.047609 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:21.047611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:21.047612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:21.047720 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:21.048040 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:21.048040 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:21.048041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:21.048351 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:21.048435 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:21.048806 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 00:35:21.048946 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":922,"thread_start_us":73,"threads_started":1}
I20260809 00:35:21.049212 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.035701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:43301)
0809 00:35:21.035939 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.035995 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:21.036724 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:21.037483 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:21.037488 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:21.037861 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:21.038126 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.038132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.039772 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.039774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.041041 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.041044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.042061 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.042062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.042218 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.042935 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:21.042949 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:21.044885 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:21.047173 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:21.047180 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:21.047182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:21.047521 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:21.047809 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:21.047810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:21.047811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:21.047964 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:21.048434 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:21.048441 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:21.048616 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 00:35:21.048746 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 00:35:21.048955 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":93,"threads_started":1}
I20260809 00:35:21.050017 3744 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45035
I20260809 00:35:21.050114 3744 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:21.050294 3744 heartbeater.cc:511] Master 127.21.41.254:45035 requested a full tablet report, sending...
I20260809 00:35:21.051118 3554 ts_manager.cc:194] Registered new tserver with Master: 55efadde246743129db9b2632ce02bac (127.21.41.193:34601)
I20260809 00:35:21.052430 3554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:43301
I20260809 00:35:21.053428 3744 heartbeater.cc:503] Master 127.21.41.254:45035 was elected leader, sending a full tablet report...
I20260809 00:35:21.057653 3752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.052068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.21.41.193:34601)
0809 00:35:21.052231 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:21.052235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:21.054580 (+ 2345us) server_negotiation.cc:407] Connection header received
0809 00:35:21.054716 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:21.054718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:21.054733 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:21.054783 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:21.055047 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.055704 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.056722 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.057198 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.057296 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.057357 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:21.057543 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 00:35:21.057585 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260809 00:35:21.057653 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.051807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:34601 (local address 127.0.0.1:59890)
0809 00:35:21.054474 (+ 2667us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.054515 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:21.054619 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:21.054880 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:21.054883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:21.054885 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:21.054948 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.054950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.055802 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.055803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.056621 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.056624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.057304 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.057305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.057393 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.057406 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 00:35:21.057516 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 00:35:21.057577 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2634,"thread_start_us":105,"threads_started":1}
I20260809 00:35:21.058099 3677 tablet_service.cc:1514] Processing DeleteTablet for tablet 1842dbf5156b471789862fe01a9d865e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:20 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:59890
I20260809 00:35:21.058670 3753 tablet_replica.cc:330] stopping tablet replica
I20260809 00:35:21.058779 3753 raft_consensus.cc:2229] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:35:21.058859 3753 raft_consensus.cc:2258] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:35:21.059176 3753 ts_tablet_manager.cc:1905] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:35:21.061370 3753 ts_tablet_manager.cc:1918] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:35:21.061435 3753 log.cc:1196] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal/wals/1842dbf5156b471789862fe01a9d865e
I20260809 00:35:21.061688 3753 ts_tablet_manager.cc:1939] T 1842dbf5156b471789862fe01a9d865e P 55efadde246743129db9b2632ce02bac: Deleting consensus metadata
I20260809 00:35:21.062263 3538 catalog_manager.cc:4841] TS 55efadde246743129db9b2632ce02bac (127.21.41.193:34601): tablet 1842dbf5156b471789862fe01a9d865e (table test-table [id=23dfb0c3418b4d15a76108448f951b9f]) successfully deleted
W20260809 00:35:21.129393 3749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:21.129639 3749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:21.129678 3749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:21.130867 3749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:21.130936 3749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:21.130980 3749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:35:21.132445 3749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:40541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=44871
--tserver_master_addrs=127.21.41.254:45035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:21.132722 3749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:21.132972 3749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:21.137970 3761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.137970 3759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.138006 3758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:21.138484 3749 server_base.cc:1034] running on GCE node
I20260809 00:35:21.138674 3749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:21.138880 3749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:21.140041 3749 hybrid_clock.cc:648] HybridClock initialized: now 1786235721140023 us; error 28 us; skew 500 ppm
Aug 09 00:35:21 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:21.142617 3749 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:35:21.143056 3749 webserver.cc:460] Webserver started at http://127.21.41.194:44871/ using document root <none> and password file <none>
I20260809 00:35:21.143292 3749 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:21.143376 3749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:21.144697 3749 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:21.145248 3768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.145422 3749 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.145517 3749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "ab7ecb09d7c24b3caddde3419f7455bd"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "da23c5cb50295a0a184bee3abb2c90ce"
server_key_iv: "d5ffb61f26138d0193ac343973c0b2ae"
server_key_version: "encryptionkey@0"
I20260809 00:35:21.145859 3749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:21.170305 3749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:21.172261 3749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:21.172426 3749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:21.172662 3749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:21.172972 3749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:21.173024 3749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.173069 3749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:21.173095 3749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.178896 3749 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:40541
I20260809 00:35:21.178956 3882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:40541 every 8 connection(s)
I20260809 00:35:21.179363 3749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:35:21 dist-test-slave-zpfg krb5kdc[2966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:21.188792 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.180148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:46993 (local address 127.21.41.254:45035)
0809 00:35:21.180220 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:21.180222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:21.180421 (+ 199us) server_negotiation.cc:407] Connection header received
0809 00:35:21.180917 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:21.180918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:21.180930 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:21.180986 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:21.181890 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.182405 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.183505 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.183711 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.183784 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.186008 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:21.186009 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:21.186010 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:21.186017 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:21.187520 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:21.187934 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:21.187935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:21.187936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:21.187992 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:21.188208 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:21.188209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:21.188209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:21.188314 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:21.188363 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:21.188596 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 00:35:21.188724 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 00:35:21.188938 3885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.180133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:46993)
0809 00:35:21.180289 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.180334 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:21.180828 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:21.181039 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:21.181043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:21.181603 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:21.181798 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.181804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.182484 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.182491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.183405 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.183408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.183798 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.183799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.183908 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.184399 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:21.184407 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:21.185916 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:21.187613 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:21.187619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:21.187621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:21.187861 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:21.188056 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:21.188056 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:21.188057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:21.188154 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:21.188367 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:21.188371 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:21.188487 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 00:35:21.188570 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:35:21.188694 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260809 00:35:21.189445 3883 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45035
I20260809 00:35:21.189546 3883 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:21.189602 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 3749
I20260809 00:35:21.189762 3883 heartbeater.cc:511] Master 127.21.41.254:45035 requested a full tablet report, sending...
I20260809 00:35:21.189889 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:43563
--local_ip_for_outbound_sockets=127.21.41.195
--tserver_master_addrs=127.21.41.254:45035
--webserver_port=40209
--webserver_interface=127.21.41.195
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:35:21.190137 3554 ts_manager.cc:194] Registered new tserver with Master: ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541)
I20260809 00:35:21.190595 3554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:46993
W20260809 00:35:21.274268 3886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:21.274507 3886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:21.274530 3886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:21.275750 3886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:21.275801 3886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:21.275828 3886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:35:21.277374 3886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:43563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=40209
--tserver_master_addrs=127.21.41.254:45035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:21.277645 3886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:21.277932 3886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:21.283138 3894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.283135 3891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.283180 3892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:21.283707 3886 server_base.cc:1034] running on GCE node
I20260809 00:35:21.283962 3886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:21.284221 3886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:21.285418 3886 hybrid_clock.cc:648] HybridClock initialized: now 1786235721285400 us; error 32 us; skew 500 ppm
Aug 09 00:35:21 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:21.288169 3886 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:35:21.288653 3886 webserver.cc:460] Webserver started at http://127.21.41.195:40209/ using document root <none> and password file <none>
I20260809 00:35:21.288848 3886 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:21.288900 3886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:21.290191 3886 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:21.290845 3901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.291024 3886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.291111 3886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "e07435db7b0e4b55810de6a80af41504"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "7212e622a44e766d2d7e7bcf5bc04193"
server_key_iv: "1a1245af8612ddfb3c80926e766df019"
server_key_version: "encryptionkey@0"
I20260809 00:35:21.291460 3886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:21.299036 3886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:21.301425 3886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:21.301594 3886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:21.301841 3886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:21.302142 3886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:21.302175 3886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.302219 3886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:21.302248 3886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.308800 3886 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:43563
I20260809 00:35:21.308892 4014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:43563 every 8 connection(s)
I20260809 00:35:21.309259 3886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:35:21.315393 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 3886
Aug 09 00:35:21 dist-test-slave-zpfg krb5kdc[2966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:21.319322 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.310015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:48869 (local address 127.21.41.254:45035)
0809 00:35:21.310063 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:21.310065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:21.310281 (+ 216us) server_negotiation.cc:407] Connection header received
0809 00:35:21.310879 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:21.310880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:21.310894 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:21.310951 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:21.311940 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.312458 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.313551 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.313766 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.313855 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.316410 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:21.316411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:21.316412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:21.316420 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:21.317861 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:21.318275 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:21.318276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:21.318277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:21.318404 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:21.318652 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:21.318653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:21.318654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:21.318767 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:21.318843 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:21.319130 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 00:35:21.319212 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:35:21.319535 4017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.309993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:48869)
0809 00:35:21.310148 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.310187 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:21.310751 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:21.311022 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:21.311026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:21.311629 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:21.311835 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.311839 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.312571 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.312572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.313448 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.313450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.313867 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.313868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.314002 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.314585 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:21.314593 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:21.316314 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:21.317947 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:21.317955 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:21.317958 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:21.318193 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:21.318458 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:21.318459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:21.318459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:21.318575 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:21.318837 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:21.318843 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:21.318992 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 00:35:21.319108 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 00:35:21.319279 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
I20260809 00:35:21.320052 4015 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45035
I20260809 00:35:21.320173 4015 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:21.320472 4015 heartbeater.cc:511] Master 127.21.41.254:45035 requested a full tablet report, sending...
I20260809 00:35:21.320897 3554 ts_manager.cc:194] Registered new tserver with Master: e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563)
I20260809 00:35:21.321382 3554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:48869
I20260809 00:35:21.322397 4019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.315814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51436 (local address 127.21.41.254:45035)
0809 00:35:21.315937 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:21.315939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:21.315979 (+ 40us) server_negotiation.cc:407] Connection header received
0809 00:35:21.316065 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:21.316066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:21.316079 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:21.316154 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:21.316731 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.317209 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.317999 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.318217 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.318295 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.319538 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:21.319539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:21.319540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:21.319547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:21.321182 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:21.321558 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:21.321559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:21.321559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:21.321615 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:21.321841 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:21.321842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:21.321842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:21.321966 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:21.322145 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:21.322185 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 00:35:21.322262 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260809 00:35:21.323086 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:35:21.325695 4019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.323459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.21.41.254:45035)
0809 00:35:21.323494 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:21.323496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:21.323585 (+ 89us) server_negotiation.cc:407] Connection header received
0809 00:35:21.323641 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:21.323642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:21.323670 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:21.323723 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:35:21.323830 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.324356 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.325101 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.325284 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.325345 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.325457 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:35:21.325543 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:35:21.325598 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:21.325631 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 00:35:21.325646 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:35:21.325996 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:35:21.327723 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:35:21.330175 3677 tablet_service.cc:1467] Processing CreateTablet for tablet 78366715f0b44767ab4a4f064f26d81e (DEFAULT_TABLE table=test-table [id=21ac1a5407cd4f959703c21126fd9715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:21.330448 3677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78366715f0b44767ab4a4f064f26d81e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:21.332131 3635 tablet_bootstrap.cc:492] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: Bootstrap starting.
I20260809 00:35:21.332479 3635 tablet_bootstrap.cc:654] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:21.333017 3635 tablet_bootstrap.cc:492] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: No bootstrap required, opened a new log
I20260809 00:35:21.333086 3635 ts_tablet_manager.cc:1397] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:35:21.333191 3635 raft_consensus.cc:348] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:21.333254 3635 raft_consensus.cc:374] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:21.333295 3635 raft_consensus.cc:729] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55efadde246743129db9b2632ce02bac, State: Initialized, Role: FOLLOWER
I20260809 00:35:21.333343 3635 consensus_queue.cc:260] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:21.333372 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.329674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:40541 (local address 127.0.0.1:36362)
0809 00:35:21.329713 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.329725 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:21.329762 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:21.330167 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:21.330168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:21.330170 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:21.330217 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.330219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.331356 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.331357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.332235 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.332237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.333131 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.333132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.333188 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.333192 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:35:21.333246 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 00:35:21.333301 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 00:35:21.333385 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.329795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.21.41.194:40541)
0809 00:35:21.329984 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:21.329990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:21.330002 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:35:21.330047 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:21.330049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:21.330067 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:21.330156 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:21.330306 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.331248 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.332332 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.333066 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.333126 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.333190 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:21.333259 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 00:35:21.333300 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260809 00:35:21.333426 3635 ts_tablet_manager.cc:1428] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.334244 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.329894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38630 (local address 127.21.41.195:43563)
0809 00:35:21.330059 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:21.330062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:21.330378 (+ 316us) server_negotiation.cc:407] Connection header received
0809 00:35:21.330506 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:21.330508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:21.330523 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:21.330592 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:21.330784 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.331825 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.333079 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.333837 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.333899 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.333951 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:21.334124 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 00:35:21.334163 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260809 00:35:21.334162 3816 tablet_service.cc:1467] Processing CreateTablet for tablet 78366715f0b44767ab4a4f064f26d81e (DEFAULT_TABLE table=test-table [id=21ac1a5407cd4f959703c21126fd9715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:21.334415 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.329793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:43563 (local address 127.0.0.1:38630)
0809 00:35:21.330233 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.330242 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:21.330410 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:21.330656 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:21.330657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:21.330659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:21.330696 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.330697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.331936 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.331937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.332986 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.332989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.333986 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.333987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.334053 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.334057 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:35:21.334109 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:35:21.334152 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":119,"threads_started":1}
I20260809 00:35:21.334545 3816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78366715f0b44767ab4a4f064f26d81e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:21.335106 3949 tablet_service.cc:1467] Processing CreateTablet for tablet 78366715f0b44767ab4a4f064f26d81e (DEFAULT_TABLE table=test-table [id=21ac1a5407cd4f959703c21126fd9715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:21.335359 3949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78366715f0b44767ab4a4f064f26d81e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:21.337594 4026 tablet_bootstrap.cc:492] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: Bootstrap starting.
I20260809 00:35:21.338114 4027 tablet_bootstrap.cc:492] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504: Bootstrap starting.
I20260809 00:35:21.338392 4026 tablet_bootstrap.cc:654] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:21.338794 4027 tablet_bootstrap.cc:654] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:21.338932 4026 log.cc:824] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:21.339334 4027 log.cc:824] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:21.339721 4026 tablet_bootstrap.cc:492] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: No bootstrap required, opened a new log
I20260809 00:35:21.339802 4026 ts_tablet_manager.cc:1397] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:35:21.340122 4027 tablet_bootstrap.cc:492] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504: No bootstrap required, opened a new log
I20260809 00:35:21.340185 4027 ts_tablet_manager.cc:1397] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:35:21.341840 4026 raft_consensus.cc:348] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:21.342026 4026 raft_consensus.cc:374] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:21.342064 4026 raft_consensus.cc:729] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab7ecb09d7c24b3caddde3419f7455bd, State: Initialized, Role: FOLLOWER
I20260809 00:35:21.342104 4027 raft_consensus.cc:348] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:21.342190 4026 consensus_queue.cc:260] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:21.342242 4027 raft_consensus.cc:374] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:21.342275 4027 raft_consensus.cc:729] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07435db7b0e4b55810de6a80af41504, State: Initialized, Role: FOLLOWER
I20260809 00:35:21.342515 4026 ts_tablet_manager.cc:1428] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:35:21.342473 4027 consensus_queue.cc:260] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:21.342532 3883 heartbeater.cc:503] Master 127.21.41.254:45035 was elected leader, sending a full tablet report...
I20260809 00:35:21.342756 4027 ts_tablet_manager.cc:1428] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:35:21.342790 4015 heartbeater.cc:503] Master 127.21.41.254:45035 was elected leader, sending a full tablet report...
I20260809 00:35:21.422048 3750 raft_consensus.cc:482] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:35:21.422195 3750 raft_consensus.cc:504] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:21.422655 3750 leader_election.cc:288] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541), e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563)
I20260809 00:35:21.426012 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.422989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:37973 (local address 127.21.41.194:40541)
0809 00:35:21.423074 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:21.423077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:21.423087 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:35:21.423119 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:21.423120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:21.423131 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:21.423192 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:21.423642 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.424316 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.425229 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.425646 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.425734 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.425744 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:21.425897 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 00:35:21.425938 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 00:35:21.426047 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.423093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:44931 (local address 127.21.41.195:43563)
0809 00:35:21.423169 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:21.423171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:21.423270 (+ 99us) server_negotiation.cc:407] Connection header received
0809 00:35:21.423350 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:21.423351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:21.423364 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:21.423433 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:21.423598 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.424229 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.425283 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.425667 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.425758 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.425768 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:21.425943 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 00:35:21.425977 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 00:35:21.426054 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.422875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:40541 (local address 127.21.41.193:37973)
0809 00:35:21.422939 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.422953 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:21.423033 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:21.423215 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:21.423217 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:21.423219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:21.423553 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.423555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.424390 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.424391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.425141 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.425143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.425732 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.425733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.425810 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.425813 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:35:21.425887 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:35:21.425931 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":28}
I20260809 00:35:21.426062 4032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.422989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:43563 (local address 127.21.41.193:44931)
0809 00:35:21.423210 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.423220 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:21.423279 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:21.423452 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:21.423453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:21.423455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:21.423526 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.423528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.424314 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.424315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.425198 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:21.425201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.425777 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.425778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:21.425859 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.425862 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:35:21.425925 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 00:35:21.425967 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":141,"threads_started":1}
I20260809 00:35:21.426522 3969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78366715f0b44767ab4a4f064f26d81e" candidate_uuid: "55efadde246743129db9b2632ce02bac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07435db7b0e4b55810de6a80af41504" is_pre_election: true
I20260809 00:35:21.426553 3836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78366715f0b44767ab4a4f064f26d81e" candidate_uuid: "55efadde246743129db9b2632ce02bac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" is_pre_election: true
I20260809 00:35:21.426690 3969 raft_consensus.cc:2454] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55efadde246743129db9b2632ce02bac in term 0.
I20260809 00:35:21.426690 3836 raft_consensus.cc:2454] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55efadde246743129db9b2632ce02bac in term 0.
I20260809 00:35:21.426901 3630 leader_election.cc:302] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55efadde246743129db9b2632ce02bac, ab7ecb09d7c24b3caddde3419f7455bd; no voters:
I20260809 00:35:21.427068 3750 raft_consensus.cc:2781] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:35:21.427142 3750 raft_consensus.cc:482] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:35:21.427162 3750 raft_consensus.cc:3037] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:21.427894 3750 raft_consensus.cc:504] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:21.428031 3750 leader_election.cc:288] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 1 election: Requested vote from peers ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541), e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563)
I20260809 00:35:21.428215 3836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78366715f0b44767ab4a4f064f26d81e" candidate_uuid: "55efadde246743129db9b2632ce02bac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab7ecb09d7c24b3caddde3419f7455bd"
I20260809 00:35:21.428300 3969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78366715f0b44767ab4a4f064f26d81e" candidate_uuid: "55efadde246743129db9b2632ce02bac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07435db7b0e4b55810de6a80af41504"
I20260809 00:35:21.428301 3836 raft_consensus.cc:3037] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:21.428390 3969 raft_consensus.cc:3037] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:21.429090 3836 raft_consensus.cc:2454] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55efadde246743129db9b2632ce02bac in term 1.
I20260809 00:35:21.429098 3969 raft_consensus.cc:2454] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55efadde246743129db9b2632ce02bac in term 1.
I20260809 00:35:21.429275 3630 leader_election.cc:302] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55efadde246743129db9b2632ce02bac, ab7ecb09d7c24b3caddde3419f7455bd; no voters:
I20260809 00:35:21.429360 3750 raft_consensus.cc:2781] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:35:21.429497 3750 raft_consensus.cc:686] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 LEADER]: Becoming Leader. State: Replica: 55efadde246743129db9b2632ce02bac, State: Running, Role: LEADER
I20260809 00:35:21.429553 3750 consensus_queue.cc:237] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
W20260809 00:35:21.429885 3884 tablet.cc:2369] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:35:21.430435 3554 catalog_manager.cc:5495] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac reported cstate change: term changed from 0 to 1, leader changed from <none> to 55efadde246743129db9b2632ce02bac (127.21.41.193). New cstate: current_term: 1 leader_uuid: "55efadde246743129db9b2632ce02bac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } health_report { overall_health: HEALTHY } } }
I20260809 00:35:21.452423 3752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.449883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59902 (local address 127.21.41.193:34601)
0809 00:35:21.449970 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:21.449972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:21.449988 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:35:21.450027 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:21.450027 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:21.450045 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:21.450110 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:35:21.450232 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.450797 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.451600 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:21.451824 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:21.451882 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:21.452041 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:35:21.452142 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:35:21.452199 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:21.452289 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 00:35:21.452308 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 00:35:21.454036 3969 raft_consensus.cc:1261] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504 [term 1 FOLLOWER]: Refusing update from remote peer 55efadde246743129db9b2632ce02bac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:21.454041 3836 raft_consensus.cc:1261] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd [term 1 FOLLOWER]: Refusing update from remote peer 55efadde246743129db9b2632ce02bac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:21.454455 3750 consensus_queue.cc:1035] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:21.454648 4033 consensus_queue.cc:1035] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:21.456848 4036 mvcc.cc:205] Tried to move back new op lower bound from 7316421515072962560 to 7316421514976071680. Current Snapshot: MvccSnapshot[applied={T|T < 7316421515072962560}]
I20260809 00:35:21.457630 4037 mvcc.cc:205] Tried to move back new op lower bound from 7316421515072962560 to 7316421514976071680. Current Snapshot: MvccSnapshot[applied={T|T < 7316421515072962560}]
I20260809 00:35:21.459360 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:35:21.459461 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:35:21.460168 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Sending DeleteTablet for 3 replicas of tablet 78366715f0b44767ab4a4f064f26d81e
I20260809 00:35:21.460521 3677 tablet_service.cc:1514] Processing DeleteTablet for tablet 78366715f0b44767ab4a4f064f26d81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:21 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:59890
I20260809 00:35:21.460518 3816 tablet_service.cc:1514] Processing DeleteTablet for tablet 78366715f0b44767ab4a4f064f26d81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:21 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:36362
I20260809 00:35:21.460611 3753 tablet_replica.cc:330] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: stopping tablet replica
I20260809 00:35:21.460654 3753 raft_consensus.cc:2229] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:35:21.460723 3753 raft_consensus.cc:2258] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:35:21.460754 4045 tablet_replica.cc:330] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: stopping tablet replica
I20260809 00:35:21.460819 3949 tablet_service.cc:1514] Processing DeleteTablet for tablet 78366715f0b44767ab4a4f064f26d81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:21 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:38630
I20260809 00:35:21.460861 4045 raft_consensus.cc:2229] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:35:21.460978 4045 raft_consensus.cc:2258] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:35:21.461040 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 3593
I20260809 00:35:21.461454 4045 ts_tablet_manager.cc:1905] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:35:21.462560 4046 tablet_replica.cc:330] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504: stopping tablet replica
I20260809 00:35:21.462692 4046 raft_consensus.cc:2229] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:35:21.462831 4046 raft_consensus.cc:2258] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:35:21.463560 4045 ts_tablet_manager.cc:1918] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:35:21.463644 4045 log.cc:1196] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/78366715f0b44767ab4a4f064f26d81e
I20260809 00:35:21.463905 4045 ts_tablet_manager.cc:1939] T 78366715f0b44767ab4a4f064f26d81e P ab7ecb09d7c24b3caddde3419f7455bd: Deleting consensus metadata
I20260809 00:35:21.464574 3539 catalog_manager.cc:4841] TS ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541): tablet 78366715f0b44767ab4a4f064f26d81e (table test-table [id=21ac1a5407cd4f959703c21126fd9715]) successfully deleted
I20260809 00:35:21.466715 4046 ts_tablet_manager.cc:1905] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:35:21.467402 3538 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:34601 (error 108)
I20260809 00:35:21.467985 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.467849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:34601 (local address 127.0.0.1:59912)
0809 00:35:21.467905 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.467943 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 00:35:21.468290 4046 ts_tablet_manager.cc:1918] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:35:21.468343 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 3749
I20260809 00:35:21.468374 4046 log.cc:1196] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/78366715f0b44767ab4a4f064f26d81e
W20260809 00:35:21.468107 3538 catalog_manager.cc:4584] TS 55efadde246743129db9b2632ce02bac (127.21.41.193:34601): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 78366715f0b44767ab4a4f064f26d81e: Network error: Client connection negotiation failed: client connection to 127.21.41.193:34601: connect: Connection refused (error 111)
I20260809 00:35:21.469995 4046 ts_tablet_manager.cc:1939] T 78366715f0b44767ab4a4f064f26d81e P e07435db7b0e4b55810de6a80af41504: Deleting consensus metadata
I20260809 00:35:21.470477 3539 catalog_manager.cc:4841] TS e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563): tablet 78366715f0b44767ab4a4f064f26d81e (table test-table [id=21ac1a5407cd4f959703c21126fd9715]) successfully deleted
I20260809 00:35:21.473546 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 3886
I20260809 00:35:21.478299 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 3521
I20260809 00:35:21.483819 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45035
--webserver_interface=127.21.41.254
--webserver_port=38101
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:45035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:21.568308 4048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:21.568514 4048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:21.568538 4048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:21.569811 4048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:35:21.569861 4048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:21.569875 4048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:21.569913 4048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:35:21.569943 4048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:35:21.571465 4048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:45035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=38101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:21.571739 4048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:21.571973 4048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:35:21.577029 4048 server_base.cc:1034] running on GCE node
W20260809 00:35:21.576967 4053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.577037 4054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.576943 4056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:21.577382 4048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:21.577663 4048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:21.578822 4048 hybrid_clock.cc:648] HybridClock initialized: now 1786235721578821 us; error 36 us; skew 500 ppm
Aug 09 00:35:21 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:21.581255 4048 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:35:21.581631 4048 webserver.cc:460] Webserver started at http://127.21.41.254:38101/ using document root <none> and password file <none>
I20260809 00:35:21.581804 4048 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:21.581848 4048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:21.583233 4048 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:21.583897 4063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.584111 4048 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:21.584233 4048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "55934827fc854cbda74871c44d5de557"
format_stamp: "Formatted at 2026-08-09 00:35:19 on dist-test-slave-zpfg"
server_key: "39519b2148b45c08e925e4e2714bc535"
server_key_iv: "9a3ccf3d4b908c4b77d4b23099aa9697"
server_key_version: "encryptionkey@0"
I20260809 00:35:21.584549 4048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:21.599769 4048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:21.601529 4048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:21.601642 4048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:21.605751 4048 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:45035
I20260809 00:35:21.605834 4115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:45035 every 8 connection(s)
I20260809 00:35:21.606189 4048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:35:21.607745 4116 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:35:21.608404 4116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Bootstrap starting.
I20260809 00:35:21.609689 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 4048
I20260809 00:35:21.609977 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:34601
--local_ip_for_outbound_sockets=127.21.41.193
--tserver_master_addrs=127.21.41.254:45035
--webserver_port=33379
--webserver_interface=127.21.41.193
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:35:21.617308 4116 log.cc:824] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:21.619542 4116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:35:21.619848 4116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557: Bootstrap complete.
I20260809 00:35:21.623157 4116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:21.623500 4116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55934827fc854cbda74871c44d5de557, State: Initialized, Role: FOLLOWER
I20260809 00:35:21.623672 4116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:21.623749 4116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:35:21.623798 4116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:35:21.623853 4116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 2 FOLLOWER]: Advancing to term 3
I20260809 00:35:21.624836 4116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:21.624953 4116 leader_election.cc:302] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55934827fc854cbda74871c44d5de557; no voters:
I20260809 00:35:21.625133 4116 leader_election.cc:288] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 00:35:21.625177 4120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 00:35:21.625291 4120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [term 3 LEADER]: Becoming Leader. State: Replica: 55934827fc854cbda74871c44d5de557, State: Running, Role: LEADER
I20260809 00:35:21.625401 4116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:35:21.625386 4120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } }
I20260809 00:35:21.625716 4120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55934827fc854cbda74871c44d5de557. Latest consensus state: current_term: 3 leader_uuid: "55934827fc854cbda74871c44d5de557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } } }
I20260809 00:35:21.625804 4120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:21.625720 4121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "55934827fc854cbda74871c44d5de557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55934827fc854cbda74871c44d5de557" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45035 } } }
I20260809 00:35:21.626020 4126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:35:21.626048 4121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55934827fc854cbda74871c44d5de557 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:21.626904 4126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:35:21.627020 4126 catalog_manager.cc:1259] Loaded cluster ID: ad8ea6ed956d428f9cbe2a22c575c996
I20260809 00:35:21.627064 4126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 00:35:21.627575 4126 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8d2bd629ad google::LogMessage::Fail() at ??:0
@ 0x7f8d2bd6588a google::LogMessage::SendToLog() at ??:0
@ 0x7f8d2bd624de google::LogMessage::Flush() at ??:0
@ 0x7f8d2bd63a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8d33c830f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f8d33c8c8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8d2dc300d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8d2dc25954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8d2ea896db start_thread at ??:0
@ 0x7f8d2a86061f clone at ??:0
W20260809 00:35:21.729995 4118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:21.730288 4118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:21.730325 4118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:21.732105 4118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:21.732167 4118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:21.732209 4118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:35:21.734470 4118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:34601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=33379
--tserver_master_addrs=127.21.41.254:45035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:21.734754 4118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:21.735040 4118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:21.740700 4153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.740759 4152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.740962 4155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:21.741001 4118 server_base.cc:1034] running on GCE node
I20260809 00:35:21.741215 4118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:21.741482 4118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:21.742656 4118 hybrid_clock.cc:648] HybridClock initialized: now 1786235721742648 us; error 33 us; skew 500 ppm
Aug 09 00:35:21 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:21.745707 4118 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:35:21.746129 4118 webserver.cc:460] Webserver started at http://127.21.41.193:33379/ using document root <none> and password file <none>
I20260809 00:35:21.746430 4118 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:21.746505 4118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:21.747890 4118 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:35:21.748625 4162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.748798 4118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:35:21.748893 4118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "55efadde246743129db9b2632ce02bac"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "bf30b3e8c2a765801bad1238c9982cd2"
server_key_iv: "3c4dd410b2c69f18e176aefdc705d111"
server_key_version: "encryptionkey@0"
I20260809 00:35:21.749238 4118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:21.759850 4118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:21.762491 4118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:21.762648 4118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:21.762898 4118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:21.763371 4169 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:35:21.764801 4118 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:35:21.764869 4118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:35:21.764905 4118 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:35:21.765765 4118 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:35:21.765815 4118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:35:21.765894 4169 tablet_bootstrap.cc:492] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: Bootstrap starting.
I20260809 00:35:21.773203 4118 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:34601
I20260809 00:35:21.773423 4276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:34601 every 8 connection(s)
I20260809 00:35:21.773674 4118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:35:21.776060 4169 log.cc:824] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:21.776259 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 4118
I20260809 00:35:21.776557 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:40541
--local_ip_for_outbound_sockets=127.21.41.194
--tserver_master_addrs=127.21.41.254:45035
--webserver_port=44871
--webserver_interface=127.21.41.194
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:35:21.778985 4169 tablet_bootstrap.cc:492] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:35:21.779277 4169 tablet_bootstrap.cc:492] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: Bootstrap complete.
I20260809 00:35:21.780455 4169 ts_tablet_manager.cc:1397] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260809 00:35:21.782274 4169 raft_consensus.cc:348] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:21.782697 4169 raft_consensus.cc:729] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55efadde246743129db9b2632ce02bac, State: Initialized, Role: FOLLOWER
I20260809 00:35:21.782896 4169 consensus_queue.cc:260] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:21.783213 4169 ts_tablet_manager.cc:1428] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
W20260809 00:35:21.838871 4279 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:35:21.774472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:45147)
0809 00:35:21.774706 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.774759 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:21.775393 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:21.838623 (+ 63230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":115,"threads_started":1}
W20260809 00:35:21.839144 4277 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:45035 (0 consecutive failures): Network error: Failed to ping master at 127.21.41.254:45035: Client connection negotiation failed: client connection to 127.21.41.254:45035: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 00:35:21.839545 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.839406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:38049)
0809 00:35:21.839470 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.839519 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:35:21.839892 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.839791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:44283)
0809 00:35:21.839843 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.839872 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 00:35:21.840016 4277 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:35:21.860096 4281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:21.860292 4281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:21.860325 4281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:21.861654 4281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:21.861697 4281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:21.861724 4281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:35:21.863255 4281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:40541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=44871
--tserver_master_addrs=127.21.41.254:45035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:21.863503 4281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:21.863734 4281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:21.868762 4290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.868732 4287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.868707 4288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:21.868906 4281 server_base.cc:1034] running on GCE node
I20260809 00:35:21.869107 4281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:21.869302 4281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:21.870415 4281 hybrid_clock.cc:648] HybridClock initialized: now 1786235721870395 us; error 28 us; skew 500 ppm
Aug 09 00:35:21 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:21.872867 4281 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:35:21.873363 4281 webserver.cc:460] Webserver started at http://127.21.41.194:44871/ using document root <none> and password file <none>
I20260809 00:35:21.873608 4281 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:21.873680 4281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:21.875101 4281 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:35:21.875734 4297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.875960 4281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:21.876050 4281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "ab7ecb09d7c24b3caddde3419f7455bd"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "da23c5cb50295a0a184bee3abb2c90ce"
server_key_iv: "d5ffb61f26138d0193ac343973c0b2ae"
server_key_version: "encryptionkey@0"
I20260809 00:35:21.876428 4281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:21.890093 4281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:21.892069 4281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:21.892236 4281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:21.892489 4281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:21.892822 4281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:21.892881 4281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.892923 4281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:21.892968 4281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:21.899093 4281 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:40541
I20260809 00:35:21.899142 4411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:40541 every 8 connection(s)
I20260809 00:35:21.899482 4281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:35:21.900506 4414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.900123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:60127)
0809 00:35:21.900275 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.900356 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
W20260809 00:35:21.900980 4412 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:45035 (0 consecutive failures): Network error: Failed to ping master at 127.21.41.254:45035: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
I20260809 00:35:21.901261 4414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.901171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:37955)
0809 00:35:21.901216 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.901244 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 00:35:21.901544 4414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:21.901471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:33809)
0809 00:35:21.901509 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:21.901530 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 00:35:21.901649 4412 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:35:21.902799 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 4281
I20260809 00:35:21.903059 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:43563
--local_ip_for_outbound_sockets=127.21.41.195
--tserver_master_addrs=127.21.41.254:45035
--webserver_port=40209
--webserver_interface=127.21.41.195
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:21.985219 4415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:21.985433 4415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:21.985457 4415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:21.986615 4415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:21.986655 4415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:21.986681 4415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:35:21.988116 4415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:43563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=40209
--tserver_master_addrs=127.21.41.254:45035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:21.988363 4415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:21.988615 4415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:35:21.993896 4415 server_base.cc:1034] running on GCE node
W20260809 00:35:21.993866 4421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.993947 4420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:21.993855 4423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:21.994191 4415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:21.994483 4415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:21.995648 4415 hybrid_clock.cc:648] HybridClock initialized: now 1786235721995645 us; error 36 us; skew 500 ppm
Aug 09 00:35:21 dist-test-slave-zpfg krb5kdc[2966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:21.998114 4415 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:35:21.998593 4415 webserver.cc:460] Webserver started at http://127.21.41.195:40209/ using document root <none> and password file <none>
I20260809 00:35:21.998822 4415 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:21.998868 4415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:22.000125 4415 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:22.000702 4430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:22.000883 4415 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:22.000973 4415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "e07435db7b0e4b55810de6a80af41504"
format_stamp: "Formatted at 2026-08-09 00:35:20 on dist-test-slave-zpfg"
server_key: "7212e622a44e766d2d7e7bcf5bc04193"
server_key_iv: "1a1245af8612ddfb3c80926e766df019"
server_key_version: "encryptionkey@0"
I20260809 00:35:22.001336 4415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:22.023181 4415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:22.025269 4415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:22.025422 4415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:22.025681 4415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:22.026015 4415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:22.026072 4415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:22.026113 4415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:22.026140 4415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:22.031883 4415 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:43563
I20260809 00:35:22.031965 4544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:43563 every 8 connection(s)
I20260809 00:35:22.032334 4415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:35:22.033427 4547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:22.033053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:35347)
0809 00:35:22.033200 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:22.033277 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
W20260809 00:35:22.033843 4545 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:45035 (0 consecutive failures): Network error: Failed to ping master at 127.21.41.254:45035: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
I20260809 00:35:22.034113 4547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:22.034029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:44035)
0809 00:35:22.034071 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:22.034096 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 00:35:22.034397 4547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:22.034315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:41503)
0809 00:35:22.034366 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:22.034385 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 00:35:22.034494 4545 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:35:22.038552 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 4415
I20260809 00:35:22.840674 4549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:22.840428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:45265)
0809 00:35:22.840569 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:22.840626 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260809 00:35:22.902221 4550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:22.902007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:55159)
0809 00:35:22.902132 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:22.902171 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
I20260809 00:35:22.992878 4551 raft_consensus.cc:482] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:35:22.993002 4551 raft_consensus.cc:504] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:22.993413 4551 leader_election.cc:288] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541), e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563)
Aug 09 00:35:22 dist-test-slave-zpfg krb5kdc[2966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.195@KRBTEST.COM
Aug 09 00:35:22 dist-test-slave-zpfg krb5kdc[2966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.194@KRBTEST.COM
I20260809 00:35:23.004590 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:22.993893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:34883 (local address 127.21.41.195:43563)
0809 00:35:22.994039 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:22.994054 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:22.994072 (+ 18us) server_negotiation.cc:407] Connection header received
0809 00:35:22.994532 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:22.994535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:22.994585 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:22.994719 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:22.995460 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:22.996344 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:22.997318 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:22.997716 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:22.997854 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:23.000468 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:23.000473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:23.000474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:23.000487 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:23.002555 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:23.002976 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:23.002980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:23.002980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:23.003129 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:23.003457 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:23.003458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:23.003458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:23.003830 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:23.003973 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:23.004324 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 00:35:23.004469 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":47,"threads_started":1}
I20260809 00:35:23.004590 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:22.993727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:35863 (local address 127.21.41.194:40541)
0809 00:35:22.993940 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:22.993951 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:22.993981 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:35:22.994512 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:22.994514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:22.994561 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:22.994717 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:22.995471 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:22.996348 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:22.997327 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:22.997716 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:22.997854 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:23.000789 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:23.000793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:23.000793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:23.000806 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:23.003046 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:23.003454 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:23.003458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:23.003458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:23.003568 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:23.003755 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:23.003756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:23.003756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:23.004082 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:23.004141 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:23.004324 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 00:35:23.004468 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":90,"threads_started":1}
I20260809 00:35:23.004585 4549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:22.993601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:40541 (local address 127.21.41.193:35863)
0809 00:35:22.993677 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:22.993695 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:22.993776 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:22.994746 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:22.994749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:22.995134 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:22.995335 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:22.995341 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:22.996465 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:22.996466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:22.997217 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:22.997219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:22.997831 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:22.997832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:22.997937 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:22.998716 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:22.998720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:23.000676 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:23.003133 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:23.003134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:23.003134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:23.003350 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:23.003639 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:23.003640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:23.003640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:23.003691 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:23.004134 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:23.004134 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:23.004174 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 00:35:23.004242 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:35:23.004395 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":16}
I20260809 00:35:23.004590 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:22.993753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:43563 (local address 127.21.41.193:34883)
0809 00:35:22.993858 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:22.993869 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:22.993927 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:22.994746 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:22.994749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:22.995135 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:22.995336 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:22.995341 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:22.996450 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:22.996451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:22.997217 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:22.997220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:22.997823 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:22.997824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:22.997937 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:22.998651 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:22.998659 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:23.000296 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:23.002640 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:23.002647 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:23.002648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:23.002870 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:23.003250 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:23.003250 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:23.003251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:23.003385 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:23.003909 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:23.003917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:23.004070 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 00:35:23.004187 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 00:35:23.004451 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":183,"thread_start_us":53,"threads_started":1}
W20260809 00:35:23.005537 4164 leader_election.cc:341] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563): Not found: Tablet not found: 78366715f0b44767ab4a4f064f26d81e
W20260809 00:35:23.005697 4164 leader_election.cc:341] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541): Not found: Tablet not found: 78366715f0b44767ab4a4f064f26d81e
I20260809 00:35:23.005746 4164 leader_election.cc:302] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55efadde246743129db9b2632ce02bac; no voters: ab7ecb09d7c24b3caddde3419f7455bd, e07435db7b0e4b55810de6a80af41504
I20260809 00:35:23.005859 4551 raft_consensus.cc:2726] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:35:23.035029 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:23.034789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:41971)
0809 00:35:23.034939 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:23.034993 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260809 00:35:23.841473 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:23.841229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:47589)
0809 00:35:23.841380 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:23.841424 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260809 00:35:23.902911 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:23.902694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:53797)
0809 00:35:23.902830 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:23.902871 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260809 00:35:24.035756 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:24.035507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:44265)
0809 00:35:24.035653 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:24.035694 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260809 00:35:24.842319 4559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:24.842059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:52391)
0809 00:35:24.842206 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:24.842259 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
I20260809 00:35:24.903585 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:24.903385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:48803)
0809 00:35:24.903507 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:24.903547 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260809 00:35:24.967691 4561 raft_consensus.cc:482] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:35:24.967834 4561 raft_consensus.cc:504] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:24.968022 4561 leader_election.cc:288] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541), e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563)
W20260809 00:35:24.968436 4164 leader_election.cc:341] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541): Not found: Tablet not found: 78366715f0b44767ab4a4f064f26d81e
W20260809 00:35:24.968531 4164 leader_election.cc:341] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563): Not found: Tablet not found: 78366715f0b44767ab4a4f064f26d81e
I20260809 00:35:24.968565 4164 leader_election.cc:302] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55efadde246743129db9b2632ce02bac; no voters: ab7ecb09d7c24b3caddde3419f7455bd, e07435db7b0e4b55810de6a80af41504
I20260809 00:35:24.968663 4561 raft_consensus.cc:2726] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:35:25.036473 4562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:25.036244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:47255)
0809 00:35:25.036386 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:25.036429 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260809 00:35:25.842991 4563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:25.842769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:41951)
0809 00:35:25.842909 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:25.842944 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260809 00:35:25.904306 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:25.904067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:49693)
0809 00:35:25.904222 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:25.904254 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260809 00:35:26.037199 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:26.036982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:58507)
0809 00:35:26.037110 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:26.037153 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260809 00:35:26.705941 4566 raft_consensus.cc:482] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:35:26.706045 4566 raft_consensus.cc:504] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:26.706207 4566 leader_election.cc:288] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541), e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563)
W20260809 00:35:26.706627 4164 leader_election.cc:341] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541): Not found: Tablet not found: 78366715f0b44767ab4a4f064f26d81e
W20260809 00:35:26.706686 4164 leader_election.cc:341] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563): Not found: Tablet not found: 78366715f0b44767ab4a4f064f26d81e
I20260809 00:35:26.706733 4164 leader_election.cc:302] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55efadde246743129db9b2632ce02bac; no voters: ab7ecb09d7c24b3caddde3419f7455bd, e07435db7b0e4b55810de6a80af41504
I20260809 00:35:26.706859 4566 raft_consensus.cc:2726] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:35:26.843773 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:26.843541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:44007)
0809 00:35:26.843679 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:26.843723 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260809 00:35:26.905125 4568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:26.904752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:51535)
0809 00:35:26.905012 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:26.905070 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":60,"threads_started":1}
I20260809 00:35:27.037981 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:27.037713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:55509)
0809 00:35:27.037889 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:27.037934 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
I20260809 00:35:27.844542 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:27.844275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:32843)
0809 00:35:27.844448 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:27.844492 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
I20260809 00:35:27.905911 4571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:27.905625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:33191)
0809 00:35:27.905820 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:27.905865 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":78,"threads_started":1}
I20260809 00:35:28.038805 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:28.038565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:33129)
0809 00:35:28.038719 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:28.038755 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260809 00:35:28.845362 4573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:28.845143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:36049)
0809 00:35:28.845275 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:28.845315 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260809 00:35:28.906670 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:28.906454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:44723)
0809 00:35:28.906590 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:28.906626 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260809 00:35:29.039592 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:29.039345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:58175)
0809 00:35:29.039501 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:29.039545 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":65,"threads_started":1}
I20260809 00:35:29.846164 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:29.845914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:33207)
0809 00:35:29.846077 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:29.846118 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
I20260809 00:35:29.907313 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:29.907094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:50059)
0809 00:35:29.907228 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:29.907266 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260809 00:35:30.032135 4578 raft_consensus.cc:482] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:35:30.032254 4578 raft_consensus.cc:504] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7ecb09d7c24b3caddde3419f7455bd" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 40541 } } peers { permanent_uuid: "e07435db7b0e4b55810de6a80af41504" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 43563 } } peers { permanent_uuid: "55efadde246743129db9b2632ce02bac" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34601 } }
I20260809 00:35:30.032456 4578 leader_election.cc:288] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541), e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563)
W20260809 00:35:30.032841 4164 leader_election.cc:341] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e07435db7b0e4b55810de6a80af41504 (127.21.41.195:43563): Not found: Tablet not found: 78366715f0b44767ab4a4f064f26d81e
W20260809 00:35:30.032920 4164 leader_election.cc:341] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab7ecb09d7c24b3caddde3419f7455bd (127.21.41.194:40541): Not found: Tablet not found: 78366715f0b44767ab4a4f064f26d81e
I20260809 00:35:30.032965 4164 leader_election.cc:302] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55efadde246743129db9b2632ce02bac; no voters: ab7ecb09d7c24b3caddde3419f7455bd, e07435db7b0e4b55810de6a80af41504
I20260809 00:35:30.033048 4578 raft_consensus.cc:2726] T 78366715f0b44767ab4a4f064f26d81e P 55efadde246743129db9b2632ce02bac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:35:30.040493 4579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:30.040242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:44827)
0809 00:35:30.040410 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:30.040452 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":55,"threads_started":1}
I20260809 00:35:30.847038 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:30.846691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:40729)
0809 00:35:30.846951 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:30.846990 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":215,"threads_started":1}
I20260809 00:35:30.908008 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:30.907785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:44917)
0809 00:35:30.907932 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:30.907966 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260809 00:35:31.041294 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:31.040980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:54427)
0809 00:35:31.041213 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:31.041249 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":180,"threads_started":1}
I20260809 00:35:31.847800 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:31.847555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:37987)
0809 00:35:31.847698 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:31.847742 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260809 00:35:31.908720 4584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:31.908487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:48037)
0809 00:35:31.908629 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:31.908673 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260809 00:35:32.042052 4585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:32.041801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:37605)
0809 00:35:32.041974 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:32.042005 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":115,"threads_started":1}
I20260809 00:35:32.848505 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:32.848298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:46029)
0809 00:35:32.848429 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:32.848458 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260809 00:35:32.909449 4587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:32.909211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:44347)
0809 00:35:32.909358 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:32.909401 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260809 00:35:33.042856 4588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:33.042632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:60669)
0809 00:35:33.042759 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:33.042808 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260809 00:35:33.849311 4589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:33.849043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:37001)
0809 00:35:33.849216 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:33.849261 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":85,"threads_started":1}
I20260809 00:35:33.910223 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:33.909970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:41575)
0809 00:35:33.910120 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:33.910176 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260809 00:35:34.043550 4591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:34.043304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:36767)
0809 00:35:34.043461 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:34.043502 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260809 00:35:34.850092 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:34.849855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:49059)
0809 00:35:34.850003 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:34.850046 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260809 00:35:34.910939 4593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:34.910690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:35727)
0809 00:35:34.910850 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:34.910893 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":79,"threads_started":1}
I20260809 00:35:35.044216 4594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:35.044010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:35449)
0809 00:35:35.044130 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:35.044169 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260809 00:35:35.850829 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:35.850604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:52789)
0809 00:35:35.850738 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:35.850781 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":58,"threads_started":1}
I20260809 00:35:35.911648 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:35.911437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:49371)
0809 00:35:35.911565 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:35.911603 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
I20260809 00:35:36.044929 4597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:36.044675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:56577)
0809 00:35:36.044849 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:36.044879 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260809 00:35:36.851656 4598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:36.851412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.193:42601)
0809 00:35:36.851571 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:36.851608 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":95,"threads_started":1}
I20260809 00:35:36.912369 4599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:36.912128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.194:33319)
0809 00:35:36.912280 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:36.912319 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":63,"threads_started":1}
I20260809 00:35:37.045660 4600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:37.045416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45035 (local address 127.21.41.195:50843)
0809 00:35:37.045563 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:37.045605 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260809 00:35:37.046995 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 4118
I20260809 00:35:37.052085 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 4281
I20260809 00:35:37.056702 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 4415
2026-08-09T00:35:37Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18368 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:35:37 dist-test-slave-zpfg krb5kdc[4603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:35:37 dist-test-slave-zpfg krb5kdc[4603](info): set up 2 sockets
Aug 09 00:35:37 dist-test-slave-zpfg krb5kdc[4603](info): commencing operation
krb5kdc: starting...
W20260809 00:35:38.119875 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:35:38 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:35:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:35:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:35:38.245646 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35649
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:35649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:38.328635 4616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:38.328845 4616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:38.328868 4616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:38.330054 4616 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 00:35:38.330098 4616 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 00:35:38.330111 4616 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 00:35:38.330123 4616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:35:38.330161 4616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:38.330196 4616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:38.330219 4616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:35:38.330246 4616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:35:38.331753 4616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:35649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:38.331997 4616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:38.332254 4616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:35:38.337656 4616 server_base.cc:1034] running on GCE node
W20260809 00:35:38.337608 4621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:38.337611 4622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:38.337625 4624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:38.338189 4616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:38.338547 4616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:38.339713 4616 hybrid_clock.cc:648] HybridClock initialized: now 1786235738339676 us; error 67 us; skew 500 ppm
Aug 09 00:35:38 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235738, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:38.347959 4616 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:35:38.348486 4616 webserver.cc:460] Webserver started at http://127.21.41.254:41413/ using document root <none> and password file <none>
I20260809 00:35:38.348745 4616 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:38.348803 4616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:38.348932 4616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:38.350044 4616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "7826684febe94449b2c5502634287b19"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "286f9c103edc7a1cfb27a9f3de910433"
server_key_iv: "36f9bab3f5ea65def7c88e64721fde78"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.350461 4616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "7826684febe94449b2c5502634287b19"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "286f9c103edc7a1cfb27a9f3de910433"
server_key_iv: "36f9bab3f5ea65def7c88e64721fde78"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.351815 4616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:38.352541 4632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:38.352715 4616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:35:38.352834 4616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "7826684febe94449b2c5502634287b19"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "286f9c103edc7a1cfb27a9f3de910433"
server_key_iv: "36f9bab3f5ea65def7c88e64721fde78"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.352902 4616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:38.367895 4616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:38.369740 4616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:38.369899 4616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:38.374106 4616 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:35649
I20260809 00:35:38.374140 4684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:35649 every 8 connection(s)
I20260809 00:35:38.374560 4616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:35:38.375451 4685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:38.377846 4685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Bootstrap starting.
I20260809 00:35:38.378492 4685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:38.378788 4685 log.cc:824] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:38.379474 4685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: No bootstrap required, opened a new log
I20260809 00:35:38.381143 4685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:35:38.381353 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 4616
I20260809 00:35:38.381367 4685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:38.381409 4685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7826684febe94449b2c5502634287b19, State: Initialized, Role: FOLLOWER
I20260809 00:35:38.381438 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:35:38.381584 4685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:35:38.381659 21671 external_mini_cluster.cc:1428] Setting key 0245b63a14f65036d10d83d9f4bb2e19
I20260809 00:35:38.381670 4685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:35:38.381707 4685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:35:38.381760 4685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:38.383037 4685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:35:38.383168 4685 leader_election.cc:302] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7826684febe94449b2c5502634287b19; no voters:
I20260809 00:35:38.383361 4685 leader_election.cc:288] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:35:38.383427 4690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:35:38.383566 4690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 1 LEADER]: Becoming Leader. State: Replica: 7826684febe94449b2c5502634287b19, State: Running, Role: LEADER
I20260809 00:35:38.383658 4685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:35:38.383689 4690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:35:38.384989 4692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7826684febe94449b2c5502634287b19. Latest consensus state: current_term: 1 leader_uuid: "7826684febe94449b2c5502634287b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } } }
I20260809 00:35:38.385085 4692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:38.385279 4691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7826684febe94449b2c5502634287b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } } }
I20260809 00:35:38.385419 4691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: This master's current role is: LEADER
W20260809 00:35:38.385622 4705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:35:38.385712 4705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:35:38.385772 4706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:35:38.385918 4706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:35:38.387475 4706 catalog_manager.cc:1347] Generated new cluster ID: e3a1d432422849f886bc4766a7e93b4e
I20260809 00:35:38.387527 4706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:35:38.391351 4706 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:35:38.391911 4706 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 00:35:38 dist-test-slave-zpfg krb5kdc[4603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:38.396466 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.382204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51020 (local address 127.21.41.254:35649)
0809 00:35:38.382495 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:38.382509 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:38.382537 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:35:38.382637 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:38.382643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:38.382687 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:38.382845 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:38.383729 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.385279 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.386404 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.387021 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.387129 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.392783 (+ 5654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:38.392787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:38.392787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:38.392799 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:38.394774 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:38.395179 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:38.395182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:38.395183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:38.395286 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:38.395494 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:38.395495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:38.395495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:38.395763 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:38.395912 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:38.396014 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 00:35:38.396156 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":207,"threads_started":1}
I20260809 00:35:38.404820 4706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Generated new TSK 0
I20260809 00:35:38.404999 4706 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:35:38.437302 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35649
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:38.521196 4711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:38.521432 4711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:38.521474 4711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:38.522713 4711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:38.522780 4711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:38.522826 4711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:35:38.524312 4711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:38.524575 4711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:38.524820 4711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:38.530117 4719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:38.530083 4717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:38.530083 4716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:38.530167 4711 server_base.cc:1034] running on GCE node
I20260809 00:35:38.530483 4711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:38.530727 4711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:38.531917 4711 hybrid_clock.cc:648] HybridClock initialized: now 1786235738531897 us; error 48 us; skew 500 ppm
Aug 09 00:35:38 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235738, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:38.534309 4711 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:35:38.534814 4711 webserver.cc:460] Webserver started at http://127.21.41.193:46603/ using document root <none> and password file <none>
I20260809 00:35:38.535053 4711 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:38.535115 4711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:38.535267 4711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:38.536339 4711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "fcc5475d322a49eaaea0c8f7eea41aa9"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "52371a51aef097ad6bb31238fa6af751"
server_key_iv: "85b845410691cc43c7b42c089359b83d"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.536695 4711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "fcc5475d322a49eaaea0c8f7eea41aa9"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "52371a51aef097ad6bb31238fa6af751"
server_key_iv: "85b845410691cc43c7b42c089359b83d"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.538038 4711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:35:38.538864 4726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:38.539031 4711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:35:38.539129 4711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "fcc5475d322a49eaaea0c8f7eea41aa9"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "52371a51aef097ad6bb31238fa6af751"
server_key_iv: "85b845410691cc43c7b42c089359b83d"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.539209 4711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:38.550318 4711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:38.552363 4711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:38.552533 4711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:38.552789 4711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:38.553105 4711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:38.553156 4711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:38.553197 4711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:38.553225 4711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:38.559480 4711 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:39041
I20260809 00:35:38.559528 4840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:39041 every 8 connection(s)
I20260809 00:35:38.559854 4711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:35:38.563020 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 4711
I20260809 00:35:38.563165 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:35:38.563373 21671 external_mini_cluster.cc:1428] Setting key 781d307b84dabd8741993812d040dd7b
Aug 09 00:35:38 dist-test-slave-zpfg krb5kdc[4603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235738, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:38.570379 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.560550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:56041 (local address 127.21.41.254:35649)
0809 00:35:38.560612 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:38.560614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:38.560835 (+ 221us) server_negotiation.cc:407] Connection header received
0809 00:35:38.561407 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:38.561408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:38.561421 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:38.561468 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:38.562473 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.563035 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.564491 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.564708 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.564814 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.567430 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:38.567431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:38.567432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:38.567439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:38.568938 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:38.569369 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:38.569370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:38.569370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:38.569432 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:38.569687 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:38.569687 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:38.569688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:38.569810 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:38.569879 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:38.570159 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 00:35:38.570260 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:35:38.570593 4843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.560540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:56041)
0809 00:35:38.560709 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:38.560747 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:38.561255 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:38.561551 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:38.561556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:38.562136 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:38.562386 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.562393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.563133 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.563134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.564397 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.564400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.564828 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.564829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.564970 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.565511 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:38.565522 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:38.567319 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:38.569036 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:38.569044 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:38.569047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:38.569294 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:38.569499 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:38.569500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:38.569501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:38.569616 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:38.569881 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:38.569889 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:38.570030 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 00:35:38.570138 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 00:35:38.570311 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260809 00:35:38.571166 4841 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35649
I20260809 00:35:38.571281 4841 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:38.571501 4841 heartbeater.cc:511] Master 127.21.41.254:35649 requested a full tablet report, sending...
I20260809 00:35:38.572181 4649 ts_manager.cc:194] Registered new tserver with Master: fcc5475d322a49eaaea0c8f7eea41aa9 (127.21.41.193:39041)
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
I20260809 00:35:38.573057 4649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:56041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:35:38.593371 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35649
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:38.676419 4846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:38.676627 4846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:38.676651 4846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:38.677873 4846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:38.677914 4846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:38.677960 4846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:35:38.679417 4846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:38.679852 4846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:38.680138 4846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:38.686105 4854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:38.686105 4851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:38.686177 4846 server_base.cc:1034] running on GCE node
W20260809 00:35:38.686172 4852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:38.686558 4846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:38.686779 4846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:38.687924 4846 hybrid_clock.cc:648] HybridClock initialized: now 1786235738687911 us; error 30 us; skew 500 ppm
Aug 09 00:35:38 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235738, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:38.690631 4846 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:35:38.691087 4846 webserver.cc:460] Webserver started at http://127.21.41.194:38289/ using document root <none> and password file <none>
I20260809 00:35:38.691337 4846 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:38.691385 4846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:38.691551 4846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:38.692606 4846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "0d42f8f592d14fa78a70ab7f33307961"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "45e22528a7f958e1c811329ec5742349"
server_key_iv: "eee03aa425704625bb1150c360d6604e"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.693059 4846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "0d42f8f592d14fa78a70ab7f33307961"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "45e22528a7f958e1c811329ec5742349"
server_key_iv: "eee03aa425704625bb1150c360d6604e"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.694470 4846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:35:38.695329 4861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:38.695513 4846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:35:38.695694 4846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "0d42f8f592d14fa78a70ab7f33307961"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "45e22528a7f958e1c811329ec5742349"
server_key_iv: "eee03aa425704625bb1150c360d6604e"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.695782 4846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:38.706519 4846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:38.708962 4846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:38.709133 4846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:38.709378 4846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:38.709694 4846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:38.709746 4846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:38.709787 4846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:38.709823 4846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:38.715965 4846 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:34807
I20260809 00:35:38.716013 4975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:34807 every 8 connection(s)
I20260809 00:35:38.716372 4846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:35:38.718715 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 4846
I20260809 00:35:38.718828 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:35:38.719041 21671 external_mini_cluster.cc:1428] Setting key 6fc80f028dd372cbe23b18b4ef5e0963
Aug 09 00:35:38 dist-test-slave-zpfg krb5kdc[4603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235738, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:38.727538 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.717115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:49163 (local address 127.21.41.254:35649)
0809 00:35:38.717181 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:38.717184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:38.717402 (+ 218us) server_negotiation.cc:407] Connection header received
0809 00:35:38.717897 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:38.717898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:38.717908 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:38.717967 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:38.719168 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.719827 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.721392 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.721713 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.721795 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.724372 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:38.724373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:38.724374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:38.724382 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:38.725803 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:38.726246 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:38.726248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:38.726248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:38.726313 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:38.726667 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:38.726668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:38.726669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:38.726827 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:38.726905 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:38.727266 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 00:35:38.727398 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:35:38.727702 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.717104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:49163)
0809 00:35:38.717281 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:38.717319 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:38.717788 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:38.718039 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:38.718043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:38.718758 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:38.719069 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.719076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.719917 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.719918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.721300 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.721303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.721821 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.721823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.721988 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.722675 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:38.722687 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:38.724266 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:38.725894 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:38.725901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:38.725904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:38.726161 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:38.726425 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:38.726426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:38.726427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:38.726585 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:38.726905 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:38.726912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:38.727100 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 00:35:38.727232 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 00:35:38.727435 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260809 00:35:38.728428 4976 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35649
I20260809 00:35:38.728550 4976 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:38.728781 4976 heartbeater.cc:511] Master 127.21.41.254:35649 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
I20260809 00:35:38.729202 4649 ts_manager.cc:194] Registered new tserver with Master: 0d42f8f592d14fa78a70ab7f33307961 (127.21.41.194:34807)
I20260809 00:35:38.729581 4649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:49163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:35:38.749264 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35649
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:38.835074 4981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:38.835281 4981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:38.835305 4981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:38.836390 4981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:38.836426 4981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:38.836457 4981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:35:38.837785 4981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:38.838135 4981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:38.838410 4981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:38.843660 4989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:38.843684 4986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:38.843736 4987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:38.843971 4981 server_base.cc:1034] running on GCE node
I20260809 00:35:38.844189 4981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:38.844424 4981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:38.845566 4981 hybrid_clock.cc:648] HybridClock initialized: now 1786235738845553 us; error 27 us; skew 500 ppm
Aug 09 00:35:38 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235738, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:38.848219 4981 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:35:38.848690 4981 webserver.cc:460] Webserver started at http://127.21.41.195:37913/ using document root <none> and password file <none>
I20260809 00:35:38.848912 4981 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:38.848956 4981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:38.849118 4981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:35:38.850195 4981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "21df0367feb642e5bcc93196aadbf3f7"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "040919b40b7941c8c2f22d701d009ec5"
server_key_iv: "01bd6b0500825d7a9b6659b65dc346f8"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.850594 4981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "21df0367feb642e5bcc93196aadbf3f7"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "040919b40b7941c8c2f22d701d009ec5"
server_key_iv: "01bd6b0500825d7a9b6659b65dc346f8"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.851908 4981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:35:38.852641 4996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:38.852816 4981 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:35:38.852902 4981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "21df0367feb642e5bcc93196aadbf3f7"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "040919b40b7941c8c2f22d701d009ec5"
server_key_iv: "01bd6b0500825d7a9b6659b65dc346f8"
server_key_version: "encryptionkey@0"
I20260809 00:35:38.852972 4981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:38.863692 4981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:38.866058 4981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:38.866218 4981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:38.866508 4981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:38.866832 4981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:38.866865 4981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:38.866904 4981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:38.866933 4981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:38.872803 4981 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:33369
I20260809 00:35:38.872853 5109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:33369 every 8 connection(s)
I20260809 00:35:38.873168 4981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:35:38.874619 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 4981
I20260809 00:35:38.874709 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:35:38.874918 21671 external_mini_cluster.cc:1428] Setting key 2e23339e21536be2e8d8075a372ab4ef
Aug 09 00:35:38 dist-test-slave-zpfg krb5kdc[4603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235738, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:38.882694 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.873903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:37793 (local address 127.21.41.254:35649)
0809 00:35:38.873952 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:38.873955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:38.874124 (+ 169us) server_negotiation.cc:407] Connection header received
0809 00:35:38.874791 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:38.874792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:38.874807 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:38.874871 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:38.876145 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.876685 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.877728 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.877993 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.878067 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.880117 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:38.880119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:38.880119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:38.880127 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:38.881268 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:38.881666 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:38.881667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:38.881668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:38.881746 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:38.881996 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:38.881997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:38.881998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:38.882113 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:38.882180 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:38.882518 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 00:35:38.882599 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:35:38.882882 5112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.873860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:37793)
0809 00:35:38.873998 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:38.874042 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:38.874638 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:38.874945 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:38.874949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:38.875816 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:38.876060 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.876065 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.876783 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.876784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.877639 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.877641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.878074 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.878075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.878209 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.878696 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:38.878704 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:38.880024 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:38.881353 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:38.881360 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:38.881362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:38.881579 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:38.881821 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:38.881822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:38.881822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:38.881929 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:38.882183 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:38.882189 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:38.882327 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 00:35:38.882483 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 00:35:38.882645 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260809 00:35:38.883378 5110 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35649
I20260809 00:35:38.883469 5110 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:38.883616 5110 heartbeater.cc:511] Master 127.21.41.254:35649 requested a full tablet report, sending...
I20260809 00:35:38.884073 4649 ts_manager.cc:194] Registered new tserver with Master: 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195:33369)
I20260809 00:35:38.884552 4649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:37793
I20260809 00:35:38.886687 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:35:38.893710 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.888065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.21.41.254:35649)
0809 00:35:38.888121 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:38.888122 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:38.888217 (+ 95us) server_negotiation.cc:407] Connection header received
0809 00:35:38.888283 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:38.888284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:38.888293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:38.888343 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:38.888878 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.889360 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.890037 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.890215 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.890264 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.891393 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:38.891394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:38.891395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:38.891401 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:38.892764 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:38.893063 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:38.893063 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:38.893064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:38.893112 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:38.893253 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:38.893253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:38.893253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:38.893402 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:38.893481 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:38.893572 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 00:35:38.893643 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:35:38.895232 4649 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:35:38.896776 4649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:35:38.903195 5122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.899072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52778 (local address 127.21.41.194:34807)
0809 00:35:38.899315 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:38.899344 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:38.899355 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:35:38.899399 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:38.899401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:38.899420 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:38.899508 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:38.899808 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.900956 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.902189 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.902743 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.902843 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.902900 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:38.903054 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 00:35:38.903096 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":88,"threads_started":1}
I20260809 00:35:38.903235 5121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.898889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:33369 (local address 127.0.0.1:37474)
0809 00:35:38.899183 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:38.899234 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:38.899312 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:38.899620 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:38.899621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:38.899623 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:38.899703 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.899705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.900642 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.900643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.901837 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.901839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.902869 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.902870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.902934 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.902956 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 00:35:38.903043 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 00:35:38.903118 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":25,"thread_start_us":66,"threads_started":1}
I20260809 00:35:38.903213 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.898889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:34807 (local address 127.0.0.1:52778)
0809 00:35:38.899239 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:38.899249 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:38.899312 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:38.899608 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:38.899611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:38.899614 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:38.899722 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.899725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.901070 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.901071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.902112 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.902114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.902847 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.902848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.902941 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.902954 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 00:35:38.903038 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:35:38.903104 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":34,"thread_start_us":64,"threads_started":1}
I20260809 00:35:38.903394 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.899272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38910 (local address 127.21.41.193:39041)
0809 00:35:38.899462 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:38.899465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:38.899474 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:35:38.899619 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:38.899622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:38.899639 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:38.899712 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:38.899930 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.900988 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.902171 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.902907 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.902967 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.903014 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:38.903254 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 00:35:38.903298 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":80,"threads_started":1}
I20260809 00:35:38.903503 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.899086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:39041 (local address 127.0.0.1:38910)
0809 00:35:38.899405 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:38.899414 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:38.899450 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:38.899813 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:38.899814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:38.899816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:38.899851 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.899852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.901093 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.901094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.902075 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:38.902077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.903046 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.903047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:38.903109 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.903112 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:35:38.903158 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 00:35:38.903192 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":44,"threads_started":1}
I20260809 00:35:38.903941 5120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:38.898966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37474 (local address 127.21.41.195:33369)
0809 00:35:38.899158 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:38.899162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:38.899396 (+ 234us) server_negotiation.cc:407] Connection header received
0809 00:35:38.899446 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:38.899448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:38.899465 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:38.899534 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:38.899784 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.900543 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.901922 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:38.902707 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:38.902776 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:38.902832 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:38.903834 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0809 00:35:38.903881 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260809 00:35:38.904294 4775 tablet_service.cc:1467] Processing CreateTablet for tablet a8bef0e81f0641c3a183e630b0ff290b (DEFAULT_TABLE table=test-table [id=558176af22514c34b44cc929457f936d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:38.904570 4909 tablet_service.cc:1467] Processing CreateTablet for tablet a8bef0e81f0641c3a183e630b0ff290b (DEFAULT_TABLE table=test-table [id=558176af22514c34b44cc929457f936d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:38.904683 4775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8bef0e81f0641c3a183e630b0ff290b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:38.904606 5044 tablet_service.cc:1467] Processing CreateTablet for tablet a8bef0e81f0641c3a183e630b0ff290b (DEFAULT_TABLE table=test-table [id=558176af22514c34b44cc929457f936d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:38.904877 4909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8bef0e81f0641c3a183e630b0ff290b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:38.904910 5044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8bef0e81f0641c3a183e630b0ff290b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:38.907473 5124 tablet_bootstrap.cc:492] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: Bootstrap starting.
I20260809 00:35:38.908003 5125 tablet_bootstrap.cc:492] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: Bootstrap starting.
I20260809 00:35:38.908130 5124 tablet_bootstrap.cc:654] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:38.908665 5125 tablet_bootstrap.cc:654] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:38.908783 5124 log.cc:824] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:38.909184 5125 log.cc:824] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:38.909605 5124 tablet_bootstrap.cc:492] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: No bootstrap required, opened a new log
I20260809 00:35:38.909675 5124 ts_tablet_manager.cc:1397] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:35:38.909868 5125 tablet_bootstrap.cc:492] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: No bootstrap required, opened a new log
I20260809 00:35:38.909932 5125 ts_tablet_manager.cc:1397] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:35:38.909513 5126 tablet_bootstrap.cc:492] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Bootstrap starting.
I20260809 00:35:38.910933 5126 tablet_bootstrap.cc:654] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:38.911427 5126 log.cc:824] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:38.911731 5124 raft_consensus.cc:348] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:38.911929 5124 raft_consensus.cc:374] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:38.911840 5125 raft_consensus.cc:348] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:38.911981 5125 raft_consensus.cc:374] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:38.911978 5124 raft_consensus.cc:729] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d42f8f592d14fa78a70ab7f33307961, State: Initialized, Role: FOLLOWER
I20260809 00:35:38.912022 5125 raft_consensus.cc:729] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df0367feb642e5bcc93196aadbf3f7, State: Initialized, Role: FOLLOWER
I20260809 00:35:38.912107 5124 consensus_queue.cc:260] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:38.912119 5125 consensus_queue.cc:260] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:38.912405 5124 ts_tablet_manager.cc:1428] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:35:38.912405 5125 ts_tablet_manager.cc:1428] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 00:35:38.912428 4976 heartbeater.cc:503] Master 127.21.41.254:35649 was elected leader, sending a full tablet report...
I20260809 00:35:38.912519 5110 heartbeater.cc:503] Master 127.21.41.254:35649 was elected leader, sending a full tablet report...
I20260809 00:35:38.913095 5126 tablet_bootstrap.cc:492] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: No bootstrap required, opened a new log
I20260809 00:35:38.913166 5126 ts_tablet_manager.cc:1397] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 00:35:38.914836 5126 raft_consensus.cc:348] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:38.914978 5126 raft_consensus.cc:374] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:38.914999 5126 raft_consensus.cc:729] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc5475d322a49eaaea0c8f7eea41aa9, State: Initialized, Role: FOLLOWER
I20260809 00:35:38.915077 5126 consensus_queue.cc:260] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:38.915304 5126 ts_tablet_manager.cc:1428] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:35:38.915339 4841 heartbeater.cc:503] Master 127.21.41.254:35649 was elected leader, sending a full tablet report...
W20260809 00:35:38.966985 4977 tablet.cc:2369] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:35:39.060662 4842 tablet.cc:2369] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:35:39.123736 5111 tablet.cc:2369] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:35:39.162083 5132 raft_consensus.cc:482] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:35:39.162263 5132 raft_consensus.cc:504] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:39.162779 5132 leader_election.cc:288] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195:33369), 0d42f8f592d14fa78a70ab7f33307961 (127.21.41.194:34807)
I20260809 00:35:39.167163 5122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:39.163094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:50433 (local address 127.21.41.194:34807)
0809 00:35:39.163149 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:39.163151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:39.163168 (+ 17us) server_negotiation.cc:407] Connection header received
0809 00:35:39.163251 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:39.163253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:39.163266 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:39.163354 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:39.163718 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:39.164469 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:39.166168 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:39.166778 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:39.166867 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:39.166873 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:39.167050 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 00:35:39.167090 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:35:39.167163 5120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:39.163094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:42319 (local address 127.21.41.195:33369)
0809 00:35:39.163151 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:39.163153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:39.163168 (+ 15us) server_negotiation.cc:407] Connection header received
0809 00:35:39.163252 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:39.163253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:39.163264 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:39.163354 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:39.163718 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:39.164488 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:39.166168 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:39.166777 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:39.166863 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:39.166872 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:39.167050 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:35:39.167089 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:35:39.167225 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:39.162982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:33369 (local address 127.21.41.193:42319)
0809 00:35:39.163112 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:39.163125 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:39.163185 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:39.163443 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:39.163445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:39.163449 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:39.163574 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:39.163577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:39.164580 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:39.164582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:39.166063 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:39.166067 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:39.166870 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:39.166872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:39.166961 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:39.166972 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 00:35:39.167037 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 00:35:39.167100 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":94,"thread_start_us":70,"threads_started":1}
I20260809 00:35:39.167229 5134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:39.162982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:34807 (local address 127.21.41.193:50433)
0809 00:35:39.163112 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:39.163125 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:39.163185 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:39.163444 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:39.163445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:39.163448 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:39.163585 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:39.163588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:39.164582 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:39.164583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:39.166063 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:39.166067 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:39.166883 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:39.166884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:39.166961 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:39.166970 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 00:35:39.167037 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 00:35:39.167100 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":25,"thread_start_us":63,"threads_started":1}
I20260809 00:35:39.167629 5064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8bef0e81f0641c3a183e630b0ff290b" candidate_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df0367feb642e5bcc93196aadbf3f7" is_pre_election: true
I20260809 00:35:39.167811 5064 raft_consensus.cc:2454] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcc5475d322a49eaaea0c8f7eea41aa9 in term 0.
I20260809 00:35:39.167858 4929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8bef0e81f0641c3a183e630b0ff290b" candidate_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d42f8f592d14fa78a70ab7f33307961" is_pre_election: true
I20260809 00:35:39.168020 4929 raft_consensus.cc:2454] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcc5475d322a49eaaea0c8f7eea41aa9 in term 0.
I20260809 00:35:39.168138 4729 leader_election.cc:302] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df0367feb642e5bcc93196aadbf3f7, fcc5475d322a49eaaea0c8f7eea41aa9; no voters:
I20260809 00:35:39.168339 5132 raft_consensus.cc:2781] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:35:39.168394 5132 raft_consensus.cc:482] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:35:39.168431 5132 raft_consensus.cc:3037] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:39.169142 5132 raft_consensus.cc:504] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:39.169289 5132 leader_election.cc:288] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [CANDIDATE]: Term 1 election: Requested vote from peers 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195:33369), 0d42f8f592d14fa78a70ab7f33307961 (127.21.41.194:34807)
I20260809 00:35:39.169459 4929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8bef0e81f0641c3a183e630b0ff290b" candidate_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d42f8f592d14fa78a70ab7f33307961"
I20260809 00:35:39.169480 5064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8bef0e81f0641c3a183e630b0ff290b" candidate_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df0367feb642e5bcc93196aadbf3f7"
I20260809 00:35:39.169545 5064 raft_consensus.cc:3037] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:39.169543 4929 raft_consensus.cc:3037] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:39.170464 5064 raft_consensus.cc:2454] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcc5475d322a49eaaea0c8f7eea41aa9 in term 1.
I20260809 00:35:39.170464 4929 raft_consensus.cc:2454] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcc5475d322a49eaaea0c8f7eea41aa9 in term 1.
I20260809 00:35:39.170636 4729 leader_election.cc:302] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df0367feb642e5bcc93196aadbf3f7, fcc5475d322a49eaaea0c8f7eea41aa9; no voters:
I20260809 00:35:39.170729 5132 raft_consensus.cc:2781] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:35:39.170876 5132 raft_consensus.cc:686] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 LEADER]: Becoming Leader. State: Replica: fcc5475d322a49eaaea0c8f7eea41aa9, State: Running, Role: LEADER
I20260809 00:35:39.170976 5132 consensus_queue.cc:237] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:39.171870 4648 catalog_manager.cc:5495] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcc5475d322a49eaaea0c8f7eea41aa9 (127.21.41.193). New cstate: current_term: 1 leader_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } health_report { overall_health: HEALTHY } } }
I20260809 00:35:39.187227 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:39.184630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38918 (local address 127.21.41.193:39041)
0809 00:35:39.184705 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:39.184708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:39.184719 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:35:39.184800 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:39.184801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:39.184831 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:39.184914 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:35:39.185067 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:39.185601 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:39.186420 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:39.186610 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:39.186686 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:39.186882 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:35:39.186984 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:35:39.187073 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:39.187112 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 00:35:39.187134 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:35:39.189088 5064 raft_consensus.cc:1261] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7 [term 1 FOLLOWER]: Refusing update from remote peer fcc5475d322a49eaaea0c8f7eea41aa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:39.189121 4929 raft_consensus.cc:1261] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Refusing update from remote peer fcc5475d322a49eaaea0c8f7eea41aa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:39.189440 5132 consensus_queue.cc:1035] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:39.189577 5135 consensus_queue.cc:1035] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:39.192307 5139 mvcc.cc:205] Tried to move back new op lower bound from 7316421587715780608 to 7316421587644948480. Current Snapshot: MvccSnapshot[applied={T|T < 7316421587715780608}]
I20260809 00:35:39.192323 5138 mvcc.cc:205] Tried to move back new op lower bound from 7316421587715780608 to 7316421587644948480. Current Snapshot: MvccSnapshot[applied={T|T < 7316421587715780608}]
I20260809 00:35:39.192639 5140 mvcc.cc:205] Tried to move back new op lower bound from 7316421587715780608 to 7316421587644948480. Current Snapshot: MvccSnapshot[applied={T|T < 7316421587715780608}]
I20260809 00:35:39.194614 4648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:35:39.194700 4648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:35:39.195410 4648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Sending DeleteTablet for 3 replicas of tablet a8bef0e81f0641c3a183e630b0ff290b
I20260809 00:35:39.195737 4909 tablet_service.cc:1514] Processing DeleteTablet for tablet a8bef0e81f0641c3a183e630b0ff290b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:39 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:52778
I20260809 00:35:39.195763 4775 tablet_service.cc:1514] Processing DeleteTablet for tablet a8bef0e81f0641c3a183e630b0ff290b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:39 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:38910
I20260809 00:35:39.195749 5044 tablet_service.cc:1514] Processing DeleteTablet for tablet a8bef0e81f0641c3a183e630b0ff290b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:39 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:37474
I20260809 00:35:39.195974 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 4711
I20260809 00:35:39.196561 5148 tablet_replica.cc:330] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: stopping tablet replica
I20260809 00:35:39.196674 5148 raft_consensus.cc:2229] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:35:39.196786 5148 raft_consensus.cc:2258] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:35:39.196983 5150 tablet_replica.cc:330] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: stopping tablet replica
I20260809 00:35:39.197085 5150 raft_consensus.cc:2229] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:35:39.197186 5150 raft_consensus.cc:2258] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:35:39.197284 5148 ts_tablet_manager.cc:1905] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:35:39.197549 5150 ts_tablet_manager.cc:1905] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:35:39.199101 5150 ts_tablet_manager.cc:1918] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:35:39.199182 5150 log.cc:1196] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/a8bef0e81f0641c3a183e630b0ff290b
I20260809 00:35:39.199390 5148 ts_tablet_manager.cc:1918] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:35:39.199465 5148 log.cc:1196] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/a8bef0e81f0641c3a183e630b0ff290b
I20260809 00:35:39.199476 5150 ts_tablet_manager.cc:1939] T a8bef0e81f0641c3a183e630b0ff290b P 21df0367feb642e5bcc93196aadbf3f7: Deleting consensus metadata
I20260809 00:35:39.199702 5148 ts_tablet_manager.cc:1939] T a8bef0e81f0641c3a183e630b0ff290b P 0d42f8f592d14fa78a70ab7f33307961: Deleting consensus metadata
I20260809 00:35:39.199877 4635 catalog_manager.cc:4841] TS 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195:33369): tablet a8bef0e81f0641c3a183e630b0ff290b (table test-table [id=558176af22514c34b44cc929457f936d]) successfully deleted
I20260809 00:35:39.200111 4634 catalog_manager.cc:4841] TS 0d42f8f592d14fa78a70ab7f33307961 (127.21.41.194:34807): tablet a8bef0e81f0641c3a183e630b0ff290b (table test-table [id=558176af22514c34b44cc929457f936d]) successfully deleted
W20260809 00:35:39.203007 4635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:39041 (error 108)
I20260809 00:35:39.203225 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 4846
I20260809 00:35:39.203490 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:39.203372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:39041 (local address 127.0.0.1:38932)
0809 00:35:39.203419 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:39.203450 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:35:39.203683 4635 catalog_manager.cc:4584] TS fcc5475d322a49eaaea0c8f7eea41aa9 (127.21.41.193:39041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8bef0e81f0641c3a183e630b0ff290b: Network error: Client connection negotiation failed: client connection to 127.21.41.193:39041: connect: Connection refused (error 111)
I20260809 00:35:39.209018 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 4981
I20260809 00:35:39.213967 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 4616
I20260809 00:35:59.219945 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35649
--webserver_interface=127.21.41.254
--webserver_port=41413
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:35649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:35:59.306643 5152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:59.306860 5152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:59.306882 5152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:59.308032 5152 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 00:35:59.308071 5152 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 00:35:59.308085 5152 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 00:35:59.308096 5152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:35:59.308109 5152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:59.308161 5152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:59.308178 5152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:35:59.308202 5152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:35:59.309716 5152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:35649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=41413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:59.309978 5152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:59.310216 5152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:59.319129 5161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:59.319139 5159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:59.319129 5158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:59.319499 5152 server_base.cc:1034] running on GCE node
I20260809 00:35:59.319851 5152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:59.320168 5152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:59.321331 5152 hybrid_clock.cc:648] HybridClock initialized: now 1786235759321311 us; error 37 us; skew 500 ppm
Aug 09 00:35:59 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235759, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:59.328009 5152 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:35:59.328529 5152 webserver.cc:460] Webserver started at http://127.21.41.254:41413/ using document root <none> and password file <none>
I20260809 00:35:59.328775 5152 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:59.328857 5152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:59.330309 5152 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:35:59.331105 5168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:59.331358 5152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:59.331470 5152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "7826684febe94449b2c5502634287b19"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "286f9c103edc7a1cfb27a9f3de910433"
server_key_iv: "36f9bab3f5ea65def7c88e64721fde78"
server_key_version: "encryptionkey@0"
I20260809 00:35:59.331897 5152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:59.342551 5152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:59.344753 5152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:59.344926 5152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:59.349400 5152 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:35649
I20260809 00:35:59.349440 5220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:35649 every 8 connection(s)
I20260809 00:35:59.349843 5152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:35:59.351552 5221 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:35:59.352243 5221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Bootstrap starting.
I20260809 00:35:59.355708 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 5152
I20260809 00:35:59.355973 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:39041
--local_ip_for_outbound_sockets=127.21.41.193
--tserver_master_addrs=127.21.41.254:35649
--webserver_port=46603
--webserver_interface=127.21.41.193
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:35:59.361068 5221 log.cc:824] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:59.363092 5221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:35:59.363411 5221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Bootstrap complete.
I20260809 00:35:59.365732 5221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:35:59.365993 5221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7826684febe94449b2c5502634287b19, State: Initialized, Role: FOLLOWER
I20260809 00:35:59.366170 5221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:35:59.366235 5221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:35:59.366290 5221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:35:59.366362 5221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:35:59.367383 5221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:35:59.367497 5221 leader_election.cc:302] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7826684febe94449b2c5502634287b19; no voters:
I20260809 00:35:59.367688 5221 leader_election.cc:288] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:35:59.367750 5225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:35:59.367910 5225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 2 LEADER]: Becoming Leader. State: Replica: 7826684febe94449b2c5502634287b19, State: Running, Role: LEADER
I20260809 00:35:59.367964 5221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:35:59.368083 5225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:35:59.368441 5226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7826684febe94449b2c5502634287b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } } }
I20260809 00:35:59.368510 5226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:59.368705 5227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7826684febe94449b2c5502634287b19. Latest consensus state: current_term: 2 leader_uuid: "7826684febe94449b2c5502634287b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } } }
I20260809 00:35:59.368801 5227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: This master's current role is: LEADER
I20260809 00:35:59.368870 5231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:35:59.369809 5231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:35:59.369973 5231 catalog_manager.cc:1259] Loaded cluster ID: e3a1d432422849f886bc4766a7e93b4e
I20260809 00:35:59.370231 5231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:35:59.371059 5231 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:35:59.371263 5231 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Loaded TSK: 0
I20260809 00:35:59.383351 5231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Generated new TSK 1
I20260809 00:35:59.383472 5231 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:35:59.451207 5223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:59.451409 5223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:59.451433 5223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:59.452570 5223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:59.452612 5223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:59.452638 5223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:35:59.454087 5223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:39041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=46603
--tserver_master_addrs=127.21.41.254:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:59.454326 5223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:59.454638 5223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:59.460302 5248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:59.460352 5223 server_base.cc:1034] running on GCE node
W20260809 00:35:59.460294 5249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:59.460517 5251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:59.460752 5223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:59.461062 5223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:59.462270 5223 hybrid_clock.cc:648] HybridClock initialized: now 1786235759462242 us; error 56 us; skew 500 ppm
Aug 09 00:35:59 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235759, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:59.465252 5223 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:35:59.465829 5223 webserver.cc:460] Webserver started at http://127.21.41.193:46603/ using document root <none> and password file <none>
I20260809 00:35:59.466087 5223 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:59.466173 5223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:59.467607 5223 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:59.468425 5258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:59.468598 5223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:59.468705 5223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "fcc5475d322a49eaaea0c8f7eea41aa9"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "52371a51aef097ad6bb31238fa6af751"
server_key_iv: "85b845410691cc43c7b42c089359b83d"
server_key_version: "encryptionkey@0"
I20260809 00:35:59.469053 5223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:59.479144 5223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:59.481448 5223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:59.481633 5223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:59.481895 5223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:59.482450 5265 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:35:59.484369 5223 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:35:59.484431 5223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:35:59.484474 5223 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:35:59.485100 5223 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:35:59.485157 5223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:35:59.485229 5265 tablet_bootstrap.cc:492] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Bootstrap starting.
I20260809 00:35:59.492436 5223 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:39041
I20260809 00:35:59.492496 5372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:39041 every 8 connection(s)
I20260809 00:35:59.492892 5223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:35:59.496546 5265 log.cc:824] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:59.499353 5265 tablet_bootstrap.cc:492] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:35:59.499689 5265 tablet_bootstrap.cc:492] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Bootstrap complete.
I20260809 00:35:59.501072 5265 ts_tablet_manager.cc:1397] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.005s
I20260809 00:35:59.501690 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 5223
I20260809 00:35:59.501991 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:34807
--local_ip_for_outbound_sockets=127.21.41.194
--tserver_master_addrs=127.21.41.254:35649
--webserver_port=38289
--webserver_interface=127.21.41.194
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:35:59.502483 5265 raft_consensus.cc:348] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:59.502818 5265 raft_consensus.cc:729] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc5475d322a49eaaea0c8f7eea41aa9, State: Initialized, Role: FOLLOWER
I20260809 00:35:59.502960 5265 consensus_queue.cc:260] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:59.503237 5265 ts_tablet_manager.cc:1428] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Aug 09 00:35:59 dist-test-slave-zpfg krb5kdc[4603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235759, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:59.508513 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.493733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:48821)
0809 00:35:59.496866 (+ 3133us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:59.496917 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:59.497464 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:59.497950 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:59.497954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:59.498264 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:59.498511 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.498517 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.499775 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.499776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.500662 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.500665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.501393 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.501394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.501556 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.502232 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:59.502243 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:59.504167 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:59.506541 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:59.506548 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:59.506550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:59.506850 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:59.507252 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:59.507253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:59.507253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:59.507379 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:59.507890 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:59.507896 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:59.508050 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 00:35:59.508150 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 00:35:59.508315 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3057,"thread_start_us":102,"threads_started":1}
I20260809 00:35:59.508699 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.493802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:48821 (local address 127.21.41.254:35649)
0809 00:35:59.496896 (+ 3094us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.496906 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.497057 (+ 151us) server_negotiation.cc:407] Connection header received
0809 00:35:59.497701 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.497706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.497752 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.497932 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:59.498635 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.499645 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.500759 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.501263 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.501402 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.504308 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:59.504312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:59.504312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:59.504323 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:59.506440 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:59.506961 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:59.506965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:59.506965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:59.507092 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:59.507471 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:59.507472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:59.507472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:59.507812 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:59.507908 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:59.508258 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 00:35:59.508428 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3040,"thread_start_us":79,"threads_started":1}
I20260809 00:35:59.510057 5373 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35649
I20260809 00:35:59.510160 5373 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:59.510365 5373 heartbeater.cc:511] Master 127.21.41.254:35649 requested a full tablet report, sending...
I20260809 00:35:59.511204 5185 ts_manager.cc:194] Registered new tserver with Master: fcc5475d322a49eaaea0c8f7eea41aa9 (127.21.41.193:39041)
I20260809 00:35:59.512552 5185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:48821
I20260809 00:35:59.513692 5373 heartbeater.cc:503] Master 127.21.41.254:35649 was elected leader, sending a full tablet report...
I20260809 00:35:59.518743 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.512167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35432 (local address 127.21.41.193:39041)
0809 00:35:59.512314 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.512318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.514559 (+ 2241us) server_negotiation.cc:407] Connection header received
0809 00:35:59.514634 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.514637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.514654 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.514749 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:59.514916 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.516026 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.517583 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.518309 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.518429 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.518487 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:59.518630 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 00:35:59.518671 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260809 00:35:59.518746 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.511902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:39041 (local address 127.0.0.1:35432)
0809 00:35:59.514440 (+ 2538us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:59.514479 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:59.514553 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:59.514766 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:59.514769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:59.514771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:59.514838 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.514840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.516124 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.516125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.517496 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.517500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.518433 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.518434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.518518 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.518531 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 00:35:59.518604 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:35:59.518655 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2506,"thread_start_us":108,"threads_started":1}
I20260809 00:35:59.519366 5307 tablet_service.cc:1514] Processing DeleteTablet for tablet a8bef0e81f0641c3a183e630b0ff290b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:39 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:35432
I20260809 00:35:59.519961 5382 tablet_replica.cc:330] stopping tablet replica
I20260809 00:35:59.520069 5382 raft_consensus.cc:2229] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:35:59.520135 5382 raft_consensus.cc:2258] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:35:59.520442 5382 ts_tablet_manager.cc:1905] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:35:59.522979 5382 ts_tablet_manager.cc:1918] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:35:59.523059 5382 log.cc:1196] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal/wals/a8bef0e81f0641c3a183e630b0ff290b
I20260809 00:35:59.523351 5382 ts_tablet_manager.cc:1939] T a8bef0e81f0641c3a183e630b0ff290b P fcc5475d322a49eaaea0c8f7eea41aa9: Deleting consensus metadata
I20260809 00:35:59.523910 5171 catalog_manager.cc:4841] TS fcc5475d322a49eaaea0c8f7eea41aa9 (127.21.41.193:39041): tablet a8bef0e81f0641c3a183e630b0ff290b (table test-table [id=558176af22514c34b44cc929457f936d]) successfully deleted
W20260809 00:35:59.590005 5379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:59.590236 5379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:59.590273 5379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:59.591420 5379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:59.591490 5379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:59.591534 5379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:35:59.592976 5379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:34807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=38289
--tserver_master_addrs=127.21.41.254:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:59.593245 5379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:59.593492 5379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:59.598757 5387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:59.598840 5388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:59.598765 5390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:59.599248 5379 server_base.cc:1034] running on GCE node
I20260809 00:35:59.599478 5379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:59.599813 5379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:59.601014 5379 hybrid_clock.cc:648] HybridClock initialized: now 1786235759601002 us; error 27 us; skew 500 ppm
Aug 09 00:35:59 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235759, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:59.603798 5379 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:35:59.604240 5379 webserver.cc:460] Webserver started at http://127.21.41.194:38289/ using document root <none> and password file <none>
I20260809 00:35:59.604487 5379 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:59.604568 5379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:59.605911 5379 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:35:59.606526 5397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:59.606753 5379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:35:59.606856 5379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "0d42f8f592d14fa78a70ab7f33307961"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "45e22528a7f958e1c811329ec5742349"
server_key_iv: "eee03aa425704625bb1150c360d6604e"
server_key_version: "encryptionkey@0"
I20260809 00:35:59.607184 5379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:59.640277 5379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:59.642226 5379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:59.642416 5379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:59.642670 5379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:59.642976 5379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:59.643029 5379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:59.643069 5379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:59.643108 5379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:59.648990 5379 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:34807
I20260809 00:35:59.649041 5510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:34807 every 8 connection(s)
I20260809 00:35:59.649426 5379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:35:59 dist-test-slave-zpfg krb5kdc[4603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235759, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:59.658583 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.650152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:55731 (local address 127.21.41.254:35649)
0809 00:35:59.650223 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.650225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.650492 (+ 267us) server_negotiation.cc:407] Connection header received
0809 00:35:59.651044 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.651046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.651059 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.651113 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:59.652074 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.652586 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.653637 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.653845 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.653948 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.656147 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:59.656148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:59.656148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:59.656155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:59.657338 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:59.657703 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:59.657703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:59.657704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:59.657757 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:59.657980 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:59.657980 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:59.657981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:59.658082 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:59.658133 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:59.658435 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 00:35:59.658510 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:35:59.658677 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.650142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:55731)
0809 00:35:59.650324 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:59.650400 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:59.650921 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:59.651197 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:59.651202 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:59.651773 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:59.651964 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.651969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.652695 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.652697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.653518 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.653520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.653956 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.653957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.654094 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.654627 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:59.654635 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:59.656050 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:59.657412 (+ 1362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:59.657419 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:59.657420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:59.657632 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:59.657819 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:59.657819 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:59.657820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:59.657921 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:59.658136 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:59.658140 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:59.658253 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 00:35:59.658380 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 00:35:59.658518 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260809 00:35:59.659175 5511 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35649
I20260809 00:35:59.659282 5511 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:59.659467 5511 heartbeater.cc:511] Master 127.21.41.254:35649 requested a full tablet report, sending...
I20260809 00:35:59.659464 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 5379
I20260809 00:35:59.659683 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:33369
--local_ip_for_outbound_sockets=127.21.41.195
--tserver_master_addrs=127.21.41.254:35649
--webserver_port=37913
--webserver_interface=127.21.41.195
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:35:59.659847 5185 ts_manager.cc:194] Registered new tserver with Master: 0d42f8f592d14fa78a70ab7f33307961 (127.21.41.194:34807)
I20260809 00:35:59.660333 5185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:55731
W20260809 00:35:59.741370 5514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:35:59.741571 5514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:35:59.741595 5514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:35:59.742687 5514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:35:59.742720 5514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:35:59.742746 5514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:35:59.744083 5514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:33369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=37913
--tserver_master_addrs=127.21.41.254:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:35:59.744269 5514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:35:59.744436 5514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:35:59.749802 5522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:59.749877 5519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:35:59.749917 5520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:35:59.749959 5514 server_base.cc:1034] running on GCE node
I20260809 00:35:59.750383 5514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:35:59.750638 5514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:35:59.751801 5514 hybrid_clock.cc:648] HybridClock initialized: now 1786235759751785 us; error 33 us; skew 500 ppm
Aug 09 00:35:59 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235759, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:35:59.754946 5514 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:35:59.755486 5514 webserver.cc:460] Webserver started at http://127.21.41.195:37913/ using document root <none> and password file <none>
I20260809 00:35:59.755759 5514 fs_manager.cc:359] Metadata directory not provided
I20260809 00:35:59.755890 5514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:35:59.757210 5514 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:59.757756 5529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:59.757926 5514 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:59.758023 5514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "21df0367feb642e5bcc93196aadbf3f7"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "040919b40b7941c8c2f22d701d009ec5"
server_key_iv: "01bd6b0500825d7a9b6659b65dc346f8"
server_key_version: "encryptionkey@0"
I20260809 00:35:59.758404 5514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:35:59.768996 5514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:35:59.771005 5514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:35:59.771143 5514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:35:59.771394 5514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:35:59.771732 5514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:35:59.771790 5514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:59.771831 5514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:35:59.771870 5514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:35:59.777994 5514 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:33369
I20260809 00:35:59.778074 5642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:33369 every 8 connection(s)
I20260809 00:35:59.778434 5514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:35:59 dist-test-slave-zpfg krb5kdc[4603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235759, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:35:59.785254 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 5514
I20260809 00:35:59.788353 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.779162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:34929 (local address 127.21.41.254:35649)
0809 00:35:59.779228 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.779230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.779428 (+ 198us) server_negotiation.cc:407] Connection header received
0809 00:35:59.780009 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.780011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.780023 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.780076 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:59.781011 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.781518 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.782641 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.782842 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.782929 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.785156 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:59.785157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:59.785157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:59.785164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:59.786787 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:59.787239 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:59.787240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:59.787240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:59.787305 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:59.787568 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:59.787569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:59.787569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:59.787739 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:59.787829 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:59.788168 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 00:35:59.788260 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:35:59.788550 5645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.779147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:34929)
0809 00:35:59.779307 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:59.779344 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:59.779872 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:59.780153 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:59.780157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:59.780715 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:35:59.780909 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.780913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.781616 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.781618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.782545 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.782548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.782941 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.782942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.783072 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.783585 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:35:59.783593 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:35:59.785023 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:35:59.786894 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:59.786901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:59.786904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:59.787134 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:59.787375 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:35:59.787376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:35:59.787377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:35:59.787485 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:35:59.787823 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:35:59.787831 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:35:59.788010 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 00:35:59.788131 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 00:35:59.788301 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260809 00:35:59.789095 5643 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35649
I20260809 00:35:59.789202 5643 heartbeater.cc:464] Registering TS with master...
I20260809 00:35:59.789412 5643 heartbeater.cc:511] Master 127.21.41.254:35649 requested a full tablet report, sending...
I20260809 00:35:59.789953 5185 ts_manager.cc:194] Registered new tserver with Master: 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195:33369)
I20260809 00:35:59.790490 5185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:34929
I20260809 00:35:59.792738 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.785653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.21.41.254:35649)
0809 00:35:59.785987 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.785988 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.785994 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:35:59.786010 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.786011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.786020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.786089 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:59.786866 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.787358 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.788262 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.788555 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.788637 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.790217 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:59.790218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:59.790218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:59.790225 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:59.791717 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:59.792038 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:59.792039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:59.792039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:59.792089 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:59.792294 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:59.792294 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:59.792295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:59.792386 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:59.792503 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:59.792613 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 00:35:59.792675 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":91,"threads_started":1}
I20260809 00:35:59.793529 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:35:59.796067 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.793855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47352 (local address 127.21.41.254:35649)
0809 00:35:59.793891 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.793893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.793995 (+ 102us) server_negotiation.cc:407] Connection header received
0809 00:35:59.794055 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.794056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.794084 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.794142 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:35:59.794248 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.794763 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.795481 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.795677 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.795734 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.795859 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:35:59.795906 (+ 47us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 00:35:59.796023 (+ 117us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47352: token expired
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:35:59.796135 5647 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47352: token expired
W20260809 00:35:59.796308 5113 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.41.254:35649)
I20260809 00:35:59.801932 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.796599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47366 (local address 127.21.41.254:35649)
0809 00:35:59.796636 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.796638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.796644 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:35:59.796660 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.796661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.796669 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.796716 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:35:59.797127 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.797572 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.798243 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.798485 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.798538 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.799576 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:35:59.799577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:35:59.799577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:35:59.799583 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:35:59.801100 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:59.801390 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:59.801390 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:59.801391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:59.801442 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:35:59.801557 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:35:59.801557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:35:59.801557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:35:59.801665 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:35:59.801737 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:59.801820 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:35:59.801876 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:35:59.803234 5185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:35:59.804821 5185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:35:59.806936 5307 tablet_service.cc:1467] Processing CreateTablet for tablet 87c39b8e9c194b1c87a9f7d3b8700971 (DEFAULT_TABLE table=test-table [id=0ae56ac188ac4619bc1300391a2caa1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:59.807145 5307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c39b8e9c194b1c87a9f7d3b8700971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:59.808729 5265 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Bootstrap starting.
I20260809 00:35:59.809072 5265 tablet_bootstrap.cc:654] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:59.809706 5265 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: No bootstrap required, opened a new log
I20260809 00:35:59.809760 5265 ts_tablet_manager.cc:1397] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:35:59.809867 5265 raft_consensus.cc:348] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:59.809952 5265 raft_consensus.cc:374] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:59.810006 5265 raft_consensus.cc:729] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc5475d322a49eaaea0c8f7eea41aa9, State: Initialized, Role: FOLLOWER
I20260809 00:35:59.810047 5265 consensus_queue.cc:260] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:59.810128 5265 ts_tablet_manager.cc:1428] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:35:59.811235 5649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.806662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:34807 (local address 127.0.0.1:46992)
0809 00:35:59.806804 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:59.806814 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:59.806854 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:59.807339 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:59.807340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:59.807342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:59.807378 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.807379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.808540 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.808541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.810050 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.810052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.810998 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.810999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.811059 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.811063 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:35:59.811125 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:35:59.811161 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":103,"threads_started":1}
I20260809 00:35:59.811256 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.806954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46992 (local address 127.21.41.194:34807)
0809 00:35:59.807179 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.807184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.807196 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:35:59.807238 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.807239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.807255 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.807350 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:59.807435 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.808452 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.810140 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.810916 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.810977 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.811053 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:59.811148 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 00:35:59.811193 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":105,"threads_started":1}
I20260809 00:35:59.811920 5445 tablet_service.cc:1467] Processing CreateTablet for tablet 87c39b8e9c194b1c87a9f7d3b8700971 (DEFAULT_TABLE table=test-table [id=0ae56ac188ac4619bc1300391a2caa1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:59.812304 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.806609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:33369 (local address 127.0.0.1:48584)
0809 00:35:59.806646 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:59.806657 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:59.806706 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:59.809015 (+ 2309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:59.809016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:59.809018 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:59.809048 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.809049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.810263 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.810264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.811240 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.811243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.812056 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.812057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.812119 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.812123 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:35:59.812182 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:35:59.812223 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 00:35:59.812332 5445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c39b8e9c194b1c87a9f7d3b8700971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:59.812537 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.806936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.21.41.195:33369)
0809 00:35:59.808824 (+ 1888us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.808829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.808851 (+ 22us) server_negotiation.cc:407] Connection header received
0809 00:35:59.808898 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.808900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.808915 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.809004 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:59.809104 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.810158 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.811341 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.811923 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.811981 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.812017 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:59.812417 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 00:35:59.812463 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1851,"thread_start_us":112,"threads_started":1}
I20260809 00:35:59.813220 5577 tablet_service.cc:1467] Processing CreateTablet for tablet 87c39b8e9c194b1c87a9f7d3b8700971 (DEFAULT_TABLE table=test-table [id=0ae56ac188ac4619bc1300391a2caa1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:35:59.813548 5577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c39b8e9c194b1c87a9f7d3b8700971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:35:59.815974 5653 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Bootstrap starting.
I20260809 00:35:59.816516 5653 tablet_bootstrap.cc:654] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:59.816928 5654 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Bootstrap starting.
I20260809 00:35:59.817158 5653 log.cc:824] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:59.817579 5654 tablet_bootstrap.cc:654] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Neither blocks nor log segments found. Creating new log.
I20260809 00:35:59.817981 5653 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: No bootstrap required, opened a new log
I20260809 00:35:59.818058 5653 ts_tablet_manager.cc:1397] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:35:59.818166 5654 log.cc:824] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:35:59.818986 5654 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: No bootstrap required, opened a new log
I20260809 00:35:59.819060 5654 ts_tablet_manager.cc:1397] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:35:59.820276 5653 raft_consensus.cc:348] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:59.820470 5653 raft_consensus.cc:374] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:59.820508 5653 raft_consensus.cc:729] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d42f8f592d14fa78a70ab7f33307961, State: Initialized, Role: FOLLOWER
I20260809 00:35:59.820627 5653 consensus_queue.cc:260] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:59.820936 5653 ts_tablet_manager.cc:1428] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:35:59.820948 5511 heartbeater.cc:503] Master 127.21.41.254:35649 was elected leader, sending a full tablet report...
I20260809 00:35:59.821077 5654 raft_consensus.cc:348] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:59.821247 5654 raft_consensus.cc:374] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:35:59.821290 5654 raft_consensus.cc:729] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df0367feb642e5bcc93196aadbf3f7, State: Initialized, Role: FOLLOWER
I20260809 00:35:59.821409 5654 consensus_queue.cc:260] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:59.821717 5643 heartbeater.cc:503] Master 127.21.41.254:35649 was elected leader, sending a full tablet report...
I20260809 00:35:59.821938 5654 ts_tablet_manager.cc:1428] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260809 00:35:59.899972 5512 tablet.cc:2369] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:35:59.949014 5658 raft_consensus.cc:482] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:35:59.949134 5658 raft_consensus.cc:504] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:59.949611 5658 leader_election.cc:288] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d42f8f592d14fa78a70ab7f33307961 (127.21.41.194:34807), fcc5475d322a49eaaea0c8f7eea41aa9 (127.21.41.193:39041)
I20260809 00:35:59.952843 5645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.949834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:34807 (local address 127.21.41.195:54107)
0809 00:35:59.949898 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:59.949911 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:59.949958 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:59.950186 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:59.950187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:59.950189 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:59.950268 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.950270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.951218 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.951219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.952002 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.952004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.952572 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.952573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.952653 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.952656 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:35:59.952707 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:35:59.952759 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":26}
I20260809 00:35:59.952867 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.949900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:54107 (local address 127.21.41.194:34807)
0809 00:35:59.949950 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.949952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.949964 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:35:59.950068 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.950069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.950080 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.950156 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:59.950372 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.951120 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.952099 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.952485 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.952543 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.952549 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:59.952734 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 00:35:59.952782 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:35:59.952917 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.949926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:39675 (local address 127.21.41.193:39041)
0809 00:35:59.949979 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.949980 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.950123 (+ 143us) server_negotiation.cc:407] Connection header received
0809 00:35:59.950155 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.950155 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.950164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.950226 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:35:59.950368 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.951120 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.952146 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.952512 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.952608 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.952617 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:59.952788 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 00:35:59.952836 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:35:59.952927 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.949835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:39041 (local address 127.21.41.195:39675)
0809 00:35:59.950040 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 00:35:59.950049 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:35:59.950104 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:35:59.950218 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:35:59.950219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:35:59.950220 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:35:59.950272 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.950273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.951231 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.951232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.952014 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:35:59.952016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.952630 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.952631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:35:59.952685 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.952688 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:35:59.952765 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:35:59.952801 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":16,"thread_start_us":117,"threads_started":1}
I20260809 00:35:59.953383 5465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c39b8e9c194b1c87a9f7d3b8700971" candidate_uuid: "21df0367feb642e5bcc93196aadbf3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d42f8f592d14fa78a70ab7f33307961" is_pre_election: true
I20260809 00:35:59.953547 5465 raft_consensus.cc:2454] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21df0367feb642e5bcc93196aadbf3f7 in term 0.
I20260809 00:35:59.953573 5327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c39b8e9c194b1c87a9f7d3b8700971" candidate_uuid: "21df0367feb642e5bcc93196aadbf3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" is_pre_election: true
I20260809 00:35:59.953778 5327 raft_consensus.cc:2454] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21df0367feb642e5bcc93196aadbf3f7 in term 0.
I20260809 00:35:59.953830 5531 leader_election.cc:302] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d42f8f592d14fa78a70ab7f33307961, 21df0367feb642e5bcc93196aadbf3f7; no voters:
I20260809 00:35:59.953972 5658 raft_consensus.cc:2781] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:35:59.954044 5658 raft_consensus.cc:482] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:35:59.954062 5658 raft_consensus.cc:3037] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:59.954743 5658 raft_consensus.cc:504] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:59.954885 5658 leader_election.cc:288] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [CANDIDATE]: Term 1 election: Requested vote from peers 0d42f8f592d14fa78a70ab7f33307961 (127.21.41.194:34807), fcc5475d322a49eaaea0c8f7eea41aa9 (127.21.41.193:39041)
I20260809 00:35:59.955010 5465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c39b8e9c194b1c87a9f7d3b8700971" candidate_uuid: "21df0367feb642e5bcc93196aadbf3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d42f8f592d14fa78a70ab7f33307961"
I20260809 00:35:59.955031 5327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c39b8e9c194b1c87a9f7d3b8700971" candidate_uuid: "21df0367feb642e5bcc93196aadbf3f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9"
I20260809 00:35:59.955096 5327 raft_consensus.cc:3037] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:59.955092 5465 raft_consensus.cc:3037] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:35:59.955993 5465 raft_consensus.cc:2454] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df0367feb642e5bcc93196aadbf3f7 in term 1.
I20260809 00:35:59.955993 5327 raft_consensus.cc:2454] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df0367feb642e5bcc93196aadbf3f7 in term 1.
I20260809 00:35:59.956177 5532 leader_election.cc:302] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df0367feb642e5bcc93196aadbf3f7, fcc5475d322a49eaaea0c8f7eea41aa9; no voters:
I20260809 00:35:59.956287 5658 raft_consensus.cc:2781] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:35:59.956456 5658 raft_consensus.cc:686] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 1 LEADER]: Becoming Leader. State: Replica: 21df0367feb642e5bcc93196aadbf3f7, State: Running, Role: LEADER
I20260809 00:35:59.956559 5658 consensus_queue.cc:237] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:35:59.957439 5183 catalog_manager.cc:5495] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195). New cstate: current_term: 1 leader_uuid: "21df0367feb642e5bcc93196aadbf3f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } health_report { overall_health: UNKNOWN } } }
I20260809 00:35:59.992131 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:35:59.989646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48592 (local address 127.21.41.195:33369)
0809 00:35:59.989700 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:35:59.989702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:35:59.989724 (+ 22us) server_negotiation.cc:407] Connection header received
0809 00:35:59.989793 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:35:59.989794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:35:59.989811 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:35:59.989878 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:35:59.989989 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.990508 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.991331 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:35:59.991519 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:35:59.991568 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:35:59.991768 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:35:59.991867 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:35:59.991928 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:35:59.992032 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 00:35:59.992050 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 00:35:59.993880 5374 tablet.cc:2369] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:35:59.993971 5327 raft_consensus.cc:1261] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Refusing update from remote peer 21df0367feb642e5bcc93196aadbf3f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:59.994022 5465 raft_consensus.cc:1261] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Refusing update from remote peer 21df0367feb642e5bcc93196aadbf3f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:35:59.994325 5658 consensus_queue.cc:1035] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:59.994535 5660 consensus_queue.cc:1035] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:35:59.997015 5664 mvcc.cc:205] Tried to move back new op lower bound from 7316421672932511744 to 7316421672782766080. Current Snapshot: MvccSnapshot[applied={T|T < 7316421672932511744}]
I20260809 00:35:59.997589 5666 mvcc.cc:205] Tried to move back new op lower bound from 7316421672932511744 to 7316421672782766080. Current Snapshot: MvccSnapshot[applied={T|T < 7316421672932511744}]
I20260809 00:35:59.999115 5183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:35:59.999239 5183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:00.000095 5183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Sending DeleteTablet for 3 replicas of tablet 87c39b8e9c194b1c87a9f7d3b8700971
I20260809 00:36:00.000428 5307 tablet_service.cc:1514] Processing DeleteTablet for tablet 87c39b8e9c194b1c87a9f7d3b8700971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:59 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:35432
I20260809 00:36:00.000445 5577 tablet_service.cc:1514] Processing DeleteTablet for tablet 87c39b8e9c194b1c87a9f7d3b8700971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:59 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:48584
I20260809 00:36:00.000447 5445 tablet_service.cc:1514] Processing DeleteTablet for tablet 87c39b8e9c194b1c87a9f7d3b8700971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:35:59 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:46992
I20260809 00:36:00.000524 5382 tablet_replica.cc:330] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: stopping tablet replica
I20260809 00:36:00.000574 5382 raft_consensus.cc:2229] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:00.000625 5382 raft_consensus.cc:2258] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:00.000728 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 5223
I20260809 00:36:00.000902 5672 tablet_replica.cc:330] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: stopping tablet replica
I20260809 00:36:00.001322 5672 raft_consensus.cc:2229] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:36:00.001453 5672 raft_consensus.cc:2258] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:00.001895 5672 ts_tablet_manager.cc:1905] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:36:00.004040 5672 ts_tablet_manager.cc:1918] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:00.004127 5672 log.cc:1196] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/87c39b8e9c194b1c87a9f7d3b8700971
I20260809 00:36:00.004439 5672 ts_tablet_manager.cc:1939] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Deleting consensus metadata
I20260809 00:36:00.004881 5171 catalog_manager.cc:4841] TS 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195:33369): tablet 87c39b8e9c194b1c87a9f7d3b8700971 (table test-table [id=0ae56ac188ac4619bc1300391a2caa1f]) successfully deleted
W20260809 00:36:00.008093 5171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:39041 (error 108)
I20260809 00:36:00.008539 5673 tablet_replica.cc:330] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: stopping tablet replica
I20260809 00:36:00.008656 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:00.008506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:39041 (local address 127.0.0.1:35434)
0809 00:36:00.008566 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:00.008616 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 00:36:00.008671 5673 raft_consensus.cc:2229] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:00.008805 5673 raft_consensus.cc:2258] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:00.009027 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 5379
W20260809 00:36:00.008791 5171 catalog_manager.cc:4584] TS fcc5475d322a49eaaea0c8f7eea41aa9 (127.21.41.193:39041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 87c39b8e9c194b1c87a9f7d3b8700971: Network error: Client connection negotiation failed: client connection to 127.21.41.193:39041: connect: Connection refused (error 111)
I20260809 00:36:00.015554 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 5514
I20260809 00:36:00.015806 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:00.015602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:34807 (local address 127.0.0.1:47006)
0809 00:36:00.015707 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:00.015754 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.194:34807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 00:36:00.022136 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 5152
I20260809 00:36:20.028784 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35649
--webserver_interface=127.21.41.254
--webserver_port=41413
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:35649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:20.113675 5675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:20.113914 5675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:20.113951 5675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:20.115113 5675 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 00:36:20.115177 5675 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 00:36:20.115198 5675 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 00:36:20.115216 5675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:20.115273 5675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:20.115314 5675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:20.115337 5675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:20.115367 5675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:20.116925 5675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:35649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=41413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:20.117199 5675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:20.117445 5675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:20.122844 5683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:20.122954 5675 server_base.cc:1034] running on GCE node
W20260809 00:36:20.122889 5680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:20.122860 5681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:20.123418 5675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:20.123800 5675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:20.124969 5675 hybrid_clock.cc:648] HybridClock initialized: now 1786235780124959 us; error 86 us; skew 500 ppm
Aug 09 00:36:20 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235780, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:20.132215 5675 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:36:20.132706 5675 webserver.cc:460] Webserver started at http://127.21.41.254:41413/ using document root <none> and password file <none>
I20260809 00:36:20.132947 5675 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:20.132994 5675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:20.134481 5675 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:20.135190 5690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:20.135413 5675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:20.135505 5675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "7826684febe94449b2c5502634287b19"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "286f9c103edc7a1cfb27a9f3de910433"
server_key_iv: "36f9bab3f5ea65def7c88e64721fde78"
server_key_version: "encryptionkey@0"
I20260809 00:36:20.135869 5675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:20.154322 5675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:20.156394 5675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:20.156558 5675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:20.160876 5675 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:35649
I20260809 00:36:20.160939 5742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:35649 every 8 connection(s)
I20260809 00:36:20.161312 5675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:20.162793 5743 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:36:20.163378 5743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Bootstrap starting.
I20260809 00:36:20.164206 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 5675
I20260809 00:36:20.164475 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:39041
--local_ip_for_outbound_sockets=127.21.41.193
--tserver_master_addrs=127.21.41.254:35649
--webserver_port=46603
--webserver_interface=127.21.41.193
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:36:20.172427 5743 log.cc:824] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:20.175261 5743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:36:20.175597 5743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Bootstrap complete.
I20260809 00:36:20.178220 5743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:36:20.178550 5743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7826684febe94449b2c5502634287b19, State: Initialized, Role: FOLLOWER
I20260809 00:36:20.178828 5743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:36:20.178921 5743 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:20.178974 5743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:20.179025 5743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 2 FOLLOWER]: Advancing to term 3
I20260809 00:36:20.180023 5743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:36:20.180160 5743 leader_election.cc:302] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7826684febe94449b2c5502634287b19; no voters:
I20260809 00:36:20.180368 5743 leader_election.cc:288] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 00:36:20.180408 5747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 00:36:20.180531 5747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [term 3 LEADER]: Becoming Leader. State: Replica: 7826684febe94449b2c5502634287b19, State: Running, Role: LEADER
I20260809 00:36:20.180682 5747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } }
I20260809 00:36:20.180685 5743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:20.181092 5749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7826684febe94449b2c5502634287b19. Latest consensus state: current_term: 3 leader_uuid: "7826684febe94449b2c5502634287b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } } }
I20260809 00:36:20.181185 5749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:20.181103 5748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7826684febe94449b2c5502634287b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7826684febe94449b2c5502634287b19" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35649 } } }
I20260809 00:36:20.181236 5748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:20.181403 5751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:20.182320 5751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:20.182533 5751 catalog_manager.cc:1259] Loaded cluster ID: e3a1d432422849f886bc4766a7e93b4e
I20260809 00:36:20.182612 5751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:20.183570 5751 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:20.183768 5751 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Loaded TSK: 0
I20260809 00:36:20.183826 5751 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7826684febe94449b2c5502634287b19: Loaded TSK: 1
F20260809 00:36:20.184047 5751 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff8b6c6c9ad google::LogMessage::Fail() at ??:0
@ 0x7ff8b6c6f88a google::LogMessage::SendToLog() at ??:0
@ 0x7ff8b6c6c4de google::LogMessage::Flush() at ??:0
@ 0x7ff8b6c6da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff8b93d9104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff8b93d6d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff8bebc61cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff8bebc648e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff8bebc6594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ff8beb8cc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7ff8beb96ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff8b8b3a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff8b8b2f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff8b99936db start_thread at ??:0
@ 0x7ff8b576a61f clone at ??:0
W20260809 00:36:20.282586 5745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:20.282884 5745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:20.282928 5745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:20.284083 5745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:20.284153 5745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:20.284199 5745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:36:20.285676 5745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:39041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=46603
--tserver_master_addrs=127.21.41.254:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:20.285938 5745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:20.286180 5745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:20.292434 5787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:20.292450 5785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:20.292646 5745 server_base.cc:1034] running on GCE node
W20260809 00:36:20.292671 5784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:20.292999 5745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:20.293426 5745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:20.294603 5745 hybrid_clock.cc:648] HybridClock initialized: now 1786235780294581 us; error 53 us; skew 500 ppm
Aug 09 00:36:20 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235780, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:20.297626 5745 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:36:20.298178 5745 webserver.cc:460] Webserver started at http://127.21.41.193:46603/ using document root <none> and password file <none>
I20260809 00:36:20.298489 5745 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:20.298568 5745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:20.300228 5745 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:20.301023 5794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:20.301244 5745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:20.301362 5745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "fcc5475d322a49eaaea0c8f7eea41aa9"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "52371a51aef097ad6bb31238fa6af751"
server_key_iv: "85b845410691cc43c7b42c089359b83d"
server_key_version: "encryptionkey@0"
I20260809 00:36:20.301748 5745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:20.332543 5745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:20.335103 5745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:20.335289 5745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:20.335541 5745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:20.336052 5801 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:36:20.338239 5745 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:36:20.338320 5745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:36:20.338397 5745 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:36:20.339057 5745 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:36:20.339123 5745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:36:20.339182 5801 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Bootstrap starting.
I20260809 00:36:20.346509 5745 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:39041
I20260809 00:36:20.346642 5909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:39041 every 8 connection(s)
I20260809 00:36:20.346941 5745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:36:20.350929 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 5745
I20260809 00:36:20.351143 5801 log.cc:824] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:20.351293 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:34807
--local_ip_for_outbound_sockets=127.21.41.194
--tserver_master_addrs=127.21.41.254:35649
--webserver_port=38289
--webserver_interface=127.21.41.194
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:36:20.353958 5801 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:36:20.354295 5801 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Bootstrap complete.
I20260809 00:36:20.355643 5801 ts_tablet_manager.cc:1397] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.012s
I20260809 00:36:20.357158 5801 raft_consensus.cc:348] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:36:20.357573 5801 raft_consensus.cc:729] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc5475d322a49eaaea0c8f7eea41aa9, State: Initialized, Role: FOLLOWER
I20260809 00:36:20.357784 5801 consensus_queue.cc:260] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:36:20.358094 5801 ts_tablet_manager.cc:1428] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260809 00:36:20.401348 5912 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:36:20.347674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:57949)
0809 00:36:20.347931 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:20.347974 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:20.348465 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:20.401115 (+ 52650us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":110,"threads_started":1}
W20260809 00:36:20.401607 5910 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:35649 (0 consecutive failures): Network error: Failed to ping master at 127.21.41.254:35649: Client connection negotiation failed: client connection to 127.21.41.254:35649: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 00:36:20.402088 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:20.401934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:42427)
0809 00:36:20.402007 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:20.402063 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 00:36:20.402444 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:20.402356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:45915)
0809 00:36:20.402395 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:20.402428 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 00:36:20.402561 5910 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:36:20.437281 5914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:20.437516 5914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:20.437556 5914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:20.438796 5914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:20.438860 5914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:20.438905 5914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:36:20.440415 5914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:34807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=38289
--tserver_master_addrs=127.21.41.254:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:20.440696 5914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:20.440943 5914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:20.446188 5920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:20.446375 5914 server_base.cc:1034] running on GCE node
W20260809 00:36:20.446203 5921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:20.446223 5923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:20.446678 5914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:20.447078 5914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:20.448230 5914 hybrid_clock.cc:648] HybridClock initialized: now 1786235780448207 us; error 37 us; skew 500 ppm
Aug 09 00:36:20 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235780, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:20.450656 5914 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:36:20.451124 5914 webserver.cc:460] Webserver started at http://127.21.41.194:38289/ using document root <none> and password file <none>
I20260809 00:36:20.451381 5914 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:20.451431 5914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:20.452903 5914 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:20.453609 5931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:20.453802 5914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:20.453907 5914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "0d42f8f592d14fa78a70ab7f33307961"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "45e22528a7f958e1c811329ec5742349"
server_key_iv: "eee03aa425704625bb1150c360d6604e"
server_key_version: "encryptionkey@0"
I20260809 00:36:20.454248 5914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:20.463636 5914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:20.465965 5914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:20.466130 5914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:20.466423 5914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:20.466863 5938 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:36:20.468276 5914 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:36:20.468343 5914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:36:20.468384 5914 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:36:20.469022 5914 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:36:20.469077 5914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:36:20.469131 5938 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Bootstrap starting.
I20260809 00:36:20.476016 5914 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:34807
I20260809 00:36:20.476100 6046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:34807 every 8 connection(s)
I20260809 00:36:20.476429 5914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:36:20.476990 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 5914
I20260809 00:36:20.477308 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:33369
--local_ip_for_outbound_sockets=127.21.41.195
--tserver_master_addrs=127.21.41.254:35649
--webserver_port=37913
--webserver_interface=127.21.41.195
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:36:20.477612 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:20.477114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:55107)
0809 00:36:20.477319 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:20.477424 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":109,"threads_started":1}
W20260809 00:36:20.477859 6047 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:35649 (0 consecutive failures): Network error: Failed to ping master at 127.21.41.254:35649: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
I20260809 00:36:20.478143 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:20.478042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:37663)
0809 00:36:20.478093 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:20.478120 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:36:20.478546 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:20.478458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:45843)
0809 00:36:20.478502 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:20.478525 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:36:20.478704 6047 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:36:20.481089 5938 log.cc:824] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:20.484153 5938 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:36:20.484519 5938 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Bootstrap complete.
I20260809 00:36:20.485889 5938 ts_tablet_manager.cc:1397] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.009s
I20260809 00:36:20.488260 5938 raft_consensus.cc:348] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:36:20.488704 5938 raft_consensus.cc:729] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d42f8f592d14fa78a70ab7f33307961, State: Initialized, Role: FOLLOWER
I20260809 00:36:20.488862 5938 consensus_queue.cc:260] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:36:20.489179 5938 ts_tablet_manager.cc:1428] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961: Time spent starting tablet: real 0.003s user 0.007s sys 0.000s
I20260809 00:36:20.489483 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:20.489372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:42361)
0809 00:36:20.489424 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:20.489460 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:36:20.489842 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:20.489736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:32779)
0809 00:36:20.489805 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:20.489827 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 00:36:20.562788 6050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:20.562984 6050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:20.563009 6050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:20.564086 6050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:20.564121 6050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:20.564149 6050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:36:20.565529 6050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:33369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=37913
--tserver_master_addrs=127.21.41.254:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:20.565713 6050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:20.565889 6050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:20.571070 6060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:20.571081 6057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:20.571081 6058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:20.571206 6050 server_base.cc:1034] running on GCE node
I20260809 00:36:20.571511 6050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:20.571760 6050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:20.572937 6050 hybrid_clock.cc:648] HybridClock initialized: now 1786235780572936 us; error 54 us; skew 500 ppm
Aug 09 00:36:20 dist-test-slave-zpfg krb5kdc[4603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235780, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:20.575487 6050 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:36:20.575932 6050 webserver.cc:460] Webserver started at http://127.21.41.195:37913/ using document root <none> and password file <none>
I20260809 00:36:20.576220 6050 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:20.576292 6050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:20.577688 6050 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:36:20.578274 6067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:20.578464 6050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:36:20.578569 6050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "21df0367feb642e5bcc93196aadbf3f7"
format_stamp: "Formatted at 2026-08-09 00:35:38 on dist-test-slave-zpfg"
server_key: "040919b40b7941c8c2f22d701d009ec5"
server_key_iv: "01bd6b0500825d7a9b6659b65dc346f8"
server_key_version: "encryptionkey@0"
I20260809 00:36:20.578917 6050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:20.592262 6050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:20.594121 6050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:20.594291 6050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:20.594559 6050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:20.594861 6050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:20.594914 6050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:20.594957 6050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:20.594982 6050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:20.600950 6050 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:33369
I20260809 00:36:20.601034 6181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:33369 every 8 connection(s)
I20260809 00:36:20.601402 6050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:36:20.602674 6184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:20.602197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:44103)
0809 00:36:20.602450 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:20.602524 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":87,"threads_started":1}
W20260809 00:36:20.603093 6182 heartbeater.cc:644] Failed to heartbeat to 127.21.41.254:35649 (0 consecutive failures): Network error: Failed to ping master at 127.21.41.254:35649: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
I20260809 00:36:20.603404 6184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:20.603307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:43635)
0809 00:36:20.603357 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:20.603386 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 00:36:20.603644 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 6050
I20260809 00:36:20.603777 6184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:20.603684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:48107)
0809 00:36:20.603729 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:20.603759 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 00:36:20.603883 6182 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:36:21.403228 6186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:21.402962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:54439)
0809 00:36:21.403110 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:21.403151 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260809 00:36:21.490447 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:21.490222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:59371)
0809 00:36:21.490355 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:21.490389 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260809 00:36:21.604405 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:21.604190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:47081)
0809 00:36:21.604324 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:21.604356 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260809 00:36:21.607908 6189 raft_consensus.cc:482] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:36:21.608012 6189 raft_consensus.cc:504] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:36:21.608381 6189 leader_election.cc:288] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195:33369), 0d42f8f592d14fa78a70ab7f33307961 (127.21.41.194:34807)
Aug 09 00:36:21 dist-test-slave-zpfg krb5kdc[4603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235780, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.195@KRBTEST.COM
Aug 09 00:36:21 dist-test-slave-zpfg krb5kdc[4603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235780, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.194@KRBTEST.COM
I20260809 00:36:21.619510 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:21.608666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:32941 (local address 127.21.41.194:34807)
0809 00:36:21.608814 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:21.608822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:21.608838 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:36:21.608971 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:21.608973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:21.609021 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:21.609154 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:21.609808 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:21.610820 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:21.611790 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:21.612273 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:21.612397 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:21.615743 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:21.615747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:21.615748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:21.615762 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:21.617891 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:21.618221 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:21.618224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:21.618225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:21.618396 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:21.618587 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:21.618588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:21.618588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:21.618997 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:21.619062 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:21.619248 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 00:36:21.619405 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":58,"threads_started":1}
I20260809 00:36:21.619522 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:21.608603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:47771 (local address 127.21.41.195:33369)
0809 00:36:21.608744 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:21.608769 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:21.608804 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:36:21.609372 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:21.609374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:21.609410 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:21.609543 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:21.609873 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:21.610869 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:21.611813 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:21.612299 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:21.612834 (+ 535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:21.615186 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:21.615190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:21.615190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:21.615200 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:21.617577 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:21.618020 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:21.618028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:21.618028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:21.618122 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:21.618430 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:21.618432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:21.618432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:21.618819 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:21.618914 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:21.619248 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 00:36:21.619397 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260809 00:36:21.619525 6186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:21.608498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:33369 (local address 127.21.41.193:47771)
0809 00:36:21.608546 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:21.608560 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:21.608619 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:21.609534 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:21.609536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:21.609712 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:21.609765 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:21.609768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:21.610955 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:21.610957 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:21.611699 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:21.611701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:21.612418 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:21.612419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:21.612547 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:21.613195 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:21.613198 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:21.615084 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:21.617665 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:21.617672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:21.617674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:21.617902 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:21.618223 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:21.618223 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:21.618224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:21.618332 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:21.618895 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:21.618901 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:21.619013 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 00:36:21.619140 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 00:36:21.619355 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":63}
I20260809 00:36:21.619524 6190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:21.608561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:34807 (local address 127.21.41.193:32941)
0809 00:36:21.608663 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:21.608673 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:21.608740 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:21.609200 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:21.609202 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:21.609488 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:21.609678 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:21.609683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:21.610920 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:21.610921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:21.611696 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:21.611697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:21.612416 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:21.612417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:21.612547 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:21.613177 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:21.613188 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:21.615518 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:21.617967 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:21.617968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:21.617968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:21.618136 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:21.618470 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:21.618471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:21.618471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:21.618521 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:21.619064 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:21.619065 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:21.619101 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 00:36:21.619171 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:36:21.619412 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":129,"thread_start_us":48,"threads_started":1}
W20260809 00:36:21.620414 5797 leader_election.cc:341] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195:33369): Not found: Tablet not found: 87c39b8e9c194b1c87a9f7d3b8700971
I20260809 00:36:21.620397 6000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c39b8e9c194b1c87a9f7d3b8700971" candidate_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0d42f8f592d14fa78a70ab7f33307961" is_pre_election: true
I20260809 00:36:21.620540 6000 raft_consensus.cc:2454] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcc5475d322a49eaaea0c8f7eea41aa9 in term 1.
I20260809 00:36:21.620802 5796 leader_election.cc:302] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d42f8f592d14fa78a70ab7f33307961, fcc5475d322a49eaaea0c8f7eea41aa9; no voters: 21df0367feb642e5bcc93196aadbf3f7
I20260809 00:36:21.620939 6189 raft_consensus.cc:2781] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 00:36:21.621038 6189 raft_consensus.cc:482] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:36:21.621079 6189 raft_consensus.cc:3037] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:36:21.622274 6189 raft_consensus.cc:504] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:36:21.622471 6189 leader_election.cc:288] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [CANDIDATE]: Term 2 election: Requested vote from peers 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195:33369), 0d42f8f592d14fa78a70ab7f33307961 (127.21.41.194:34807)
I20260809 00:36:21.622648 6000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c39b8e9c194b1c87a9f7d3b8700971" candidate_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0d42f8f592d14fa78a70ab7f33307961"
I20260809 00:36:21.622764 6000 raft_consensus.cc:3037] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 1 FOLLOWER]: Advancing to term 2
W20260809 00:36:21.622835 5797 leader_election.cc:341] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195:33369): Not found: Tablet not found: 87c39b8e9c194b1c87a9f7d3b8700971
I20260809 00:36:21.623934 6000 raft_consensus.cc:2454] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcc5475d322a49eaaea0c8f7eea41aa9 in term 2.
I20260809 00:36:21.624126 5796 leader_election.cc:302] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d42f8f592d14fa78a70ab7f33307961, fcc5475d322a49eaaea0c8f7eea41aa9; no voters: 21df0367feb642e5bcc93196aadbf3f7
I20260809 00:36:21.624229 6189 raft_consensus.cc:2781] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:36:21.624366 6189 raft_consensus.cc:686] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [term 2 LEADER]: Becoming Leader. State: Replica: fcc5475d322a49eaaea0c8f7eea41aa9, State: Running, Role: LEADER
I20260809 00:36:21.624480 6189 consensus_queue.cc:237] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:36:21.624708 6186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:21.624558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:50489)
0809 00:36:21.624639 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:21.624676 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 00:36:22.094822 6000 raft_consensus.cc:1261] T 87c39b8e9c194b1c87a9f7d3b8700971 P 0d42f8f592d14fa78a70ab7f33307961 [term 2 FOLLOWER]: Refusing update from remote peer fcc5475d322a49eaaea0c8f7eea41aa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 00:36:22.095229 6189 consensus_queue.cc:1035] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 00:36:22.096272 6197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:22.096064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:40955)
0809 00:36:22.096180 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:22.096226 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":49,"threads_started":1}
I20260809 00:36:22.097203 6186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:22.097099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:48593)
0809 00:36:22.097140 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:22.097166 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 00:36:22.097378 5797 consensus_peers.cc:487] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9 -> Peer 21df0367feb642e5bcc93196aadbf3f7 (127.21.41.195:33369): Couldn't send request to peer 21df0367feb642e5bcc93196aadbf3f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87c39b8e9c194b1c87a9f7d3b8700971. This is attempt 1: this message will repeat every 5th retry.
I20260809 00:36:22.097410 6197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:22.097316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:47503)
0809 00:36:22.097356 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:22.097379 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 00:36:22.573741 6200 ts_tablet_manager.cc:927] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Initiating tablet copy from peer fcc5475d322a49eaaea0c8f7eea41aa9 (127.21.41.193:39041)
I20260809 00:36:22.573989 6200 tablet_copy_client.cc:316] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: tablet copy: Beginning tablet copy session from remote peer at address 127.21.41.193:39041
Aug 09 00:36:22 dist-test-slave-zpfg krb5kdc[4603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235780, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.193@KRBTEST.COM
I20260809 00:36:22.581934 6203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:22.574467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:38173 (local address 127.21.41.193:39041)
0809 00:36:22.574601 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:22.574605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:22.574617 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:36:22.574650 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:22.574651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:22.574664 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:22.574734 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:22.575222 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:22.575946 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:22.576742 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:22.577042 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:22.577098 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:22.579153 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:22.579181 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:22.579181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:22.579191 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:22.580668 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:22.581042 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:22.581045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:22.581046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:22.581099 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:22.581315 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:22.581316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:22.581316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:22.581474 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:22.581539 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:22.581759 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 00:36:22.581862 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260809 00:36:22.581944 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:22.574319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:39041 (local address 127.21.41.195:38173)
0809 00:36:22.574540 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:22.574557 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:22.574611 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:22.574768 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:22.574771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:22.575084 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:22.575149 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:22.575151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:22.576029 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:22.576030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:22.576671 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:22.576673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:22.577111 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:22.577112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:22.577172 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:22.577632 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:22.577636 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:22.579072 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:22.580746 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:22.580753 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:22.580755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:22.580967 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:22.581178 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:22.581179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:22.581179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:22.581245 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:22.581560 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:22.581566 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:22.581613 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 00:36:22.581686 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:36:22.581860 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":152,"threads_started":1}
I20260809 00:36:22.582482 5884 tablet_copy_service.cc:140] P fcc5475d322a49eaaea0c8f7eea41aa9: Received BeginTabletCopySession request for tablet 87c39b8e9c194b1c87a9f7d3b8700971 from peer 21df0367feb642e5bcc93196aadbf3f7 ({username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:38173)
I20260809 00:36:22.582546 5884 tablet_copy_service.cc:161] P fcc5475d322a49eaaea0c8f7eea41aa9: Beginning new tablet copy session on tablet 87c39b8e9c194b1c87a9f7d3b8700971 from peer 21df0367feb642e5bcc93196aadbf3f7 at {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:38173: session id = 21df0367feb642e5bcc93196aadbf3f7-87c39b8e9c194b1c87a9f7d3b8700971
I20260809 00:36:22.583065 5884 tablet_copy_source_session.cc:205] T 87c39b8e9c194b1c87a9f7d3b8700971 P fcc5475d322a49eaaea0c8f7eea41aa9: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 00:36:22.584118 6200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c39b8e9c194b1c87a9f7d3b8700971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:22.586818 6200 tablet_copy_client.cc:791] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: tablet copy: Starting download of 0 data blocks...
I20260809 00:36:22.587001 6200 tablet_copy_client.cc:655] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: tablet copy: Starting download of 1 WAL segments...
I20260809 00:36:22.587999 6200 tablet_copy_client.cc:531] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 00:36:22.588719 6200 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Bootstrap starting.
I20260809 00:36:22.590312 6200 log.cc:824] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:22.592711 6200 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:36:22.592954 6200 tablet_bootstrap.cc:492] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Bootstrap complete.
I20260809 00:36:22.593097 6200 ts_tablet_manager.cc:1397] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 00:36:22.594086 6200 raft_consensus.cc:348] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:36:22.594229 6200 raft_consensus.cc:729] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df0367feb642e5bcc93196aadbf3f7, State: Initialized, Role: FOLLOWER
I20260809 00:36:22.594399 6200 consensus_queue.cc:260] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df0367feb642e5bcc93196aadbf3f7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 33369 } } peers { permanent_uuid: "0d42f8f592d14fa78a70ab7f33307961" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 34807 } } peers { permanent_uuid: "fcc5475d322a49eaaea0c8f7eea41aa9" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 39041 } }
I20260809 00:36:22.594687 6200 ts_tablet_manager.cc:1428] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 00:36:22.594985 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:22.594866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:35347)
0809 00:36:22.594923 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:22.594951 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:36:22.595005 5884 tablet_copy_service.cc:342] P fcc5475d322a49eaaea0c8f7eea41aa9: Request end of tablet copy session 21df0367feb642e5bcc93196aadbf3f7-87c39b8e9c194b1c87a9f7d3b8700971 received from {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:38173
I20260809 00:36:22.595057 5884 tablet_copy_service.cc:433] P fcc5475d322a49eaaea0c8f7eea41aa9: ending tablet copy session 21df0367feb642e5bcc93196aadbf3f7-87c39b8e9c194b1c87a9f7d3b8700971 on tablet 87c39b8e9c194b1c87a9f7d3b8700971 with peer 21df0367feb642e5bcc93196aadbf3f7
I20260809 00:36:22.595358 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:22.595245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:50389)
0809 00:36:22.595298 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:22.595331 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 00:36:23.097934 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:23.097627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:60607)
0809 00:36:23.097843 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:23.097883 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":155,"threads_started":1}
I20260809 00:36:23.098070 6207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:23.097820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:34649)
0809 00:36:23.097957 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:23.098009 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260809 00:36:23.105286 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:23.105136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:56637)
0809 00:36:23.105217 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:23.105247 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 00:36:23.105293 6135 raft_consensus.cc:1203] T 87c39b8e9c194b1c87a9f7d3b8700971 P 21df0367feb642e5bcc93196aadbf3f7 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260809 00:36:23.106036 6212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:23.105836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:46179)
0809 00:36:23.105955 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:23.105986 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":81,"threads_started":1}
I20260809 00:36:24.098912 6214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:24.098619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:37031)
0809 00:36:24.098787 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:24.098826 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260809 00:36:24.106019 6215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:24.105704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:37599)
0809 00:36:24.105899 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:24.105944 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":124,"threads_started":1}
I20260809 00:36:24.106668 6216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:24.106483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:55295)
0809 00:36:24.106594 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:24.106629 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
I20260809 00:36:25.099649 6217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:25.099449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:60789)
0809 00:36:25.099566 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:25.099602 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260809 00:36:25.106796 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:25.106529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:56899)
0809 00:36:25.106696 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:25.106743 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":61,"threads_started":1}
I20260809 00:36:25.107285 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:25.107098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:44497)
0809 00:36:25.107214 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:25.107245 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":68,"threads_started":1}
I20260809 00:36:26.100364 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:26.100135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:44005)
0809 00:36:26.100280 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:26.100319 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260809 00:36:26.107604 6221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:26.107326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:45389)
0809 00:36:26.107450 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:26.107530 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260809 00:36:26.108052 6221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:26.107935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:33705)
0809 00:36:26.107992 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:26.108025 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:36:26.108073 6222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:26.107828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:52973)
0809 00:36:26.107998 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:26.108031 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":81,"threads_started":1}
I20260809 00:36:27.101001 6223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:27.100772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:36499)
0809 00:36:27.100910 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:27.100951 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":59,"threads_started":1}
I20260809 00:36:27.108732 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:27.108502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:45087)
0809 00:36:27.108647 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:27.108685 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260809 00:36:27.108732 6225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:27.108513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:47611)
0809 00:36:27.108647 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:27.108684 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260809 00:36:28.101729 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:28.101505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:51245)
0809 00:36:28.101635 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:28.101679 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260809 00:36:28.109479 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:28.109267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:49825)
0809 00:36:28.109396 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:28.109434 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260809 00:36:28.109498 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:28.109272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:41239)
0809 00:36:28.109429 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:28.109458 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260809 00:36:29.102629 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:29.102314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:48061)
0809 00:36:29.102507 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:29.102568 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":133,"threads_started":1}
I20260809 00:36:29.110270 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:29.110031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:56035)
0809 00:36:29.110178 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:29.110216 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260809 00:36:29.110273 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:29.110030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:44549)
0809 00:36:29.110188 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:29.110228 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260809 00:36:30.103381 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:30.103117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:40483)
0809 00:36:30.103245 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:30.103307 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260809 00:36:30.110956 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:30.110736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:45133)
0809 00:36:30.110884 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:30.110915 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260809 00:36:30.110952 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:30.110737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:53141)
0809 00:36:30.110864 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:30.110905 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260809 00:36:31.104192 6235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:31.103954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:60551)
0809 00:36:31.104100 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:31.104143 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260809 00:36:31.111740 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:31.111500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:60825)
0809 00:36:31.111656 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:31.111686 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260809 00:36:31.111747 6236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:31.111495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:33457)
0809 00:36:31.111645 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:31.111690 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260809 00:36:32.105001 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:32.104737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:53327)
0809 00:36:32.104915 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:32.104953 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260809 00:36:32.112634 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:32.112263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:37597)
0809 00:36:32.112540 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:32.112585 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":62,"threads_started":1}
I20260809 00:36:32.112813 6240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:32.112604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:51877)
0809 00:36:32.112731 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:32.112767 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260809 00:36:33.105934 6241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:33.105600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:46735)
0809 00:36:33.105790 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:33.105827 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
I20260809 00:36:33.113477 6242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:33.113247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:57127)
0809 00:36:33.113393 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:33.113430 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260809 00:36:33.113487 6243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:33.113270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:37517)
0809 00:36:33.113413 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:33.113446 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":58,"threads_started":1}
I20260809 00:36:34.106796 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:34.106505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:57429)
0809 00:36:34.106658 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:34.106704 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
I20260809 00:36:34.114223 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:34.113982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:37581)
0809 00:36:34.114120 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:34.114178 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260809 00:36:34.114225 6247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:34.113999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:52763)
0809 00:36:34.114144 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:34.114182 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":60,"threads_started":1}
I20260809 00:36:35.107630 6248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:35.107382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.194:55863)
0809 00:36:35.107536 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:35.107580 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
I20260809 00:36:35.115006 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:35.114752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.195:57225)
0809 00:36:35.114905 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:35.114948 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260809 00:36:35.115121 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:35.114854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35649 (local address 127.21.41.193:56313)
0809 00:36:35.115006 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:35.115051 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.254:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
I20260809 00:36:35.606308 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 5745
I20260809 00:36:35.611402 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 5914
I20260809 00:36:35.616492 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 6050
2026-08-09T00:36:35Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58560 ms)
[----------] 30 tests from SecurityITest (235771 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:35 dist-test-slave-zpfg krb5kdc[6253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:35 dist-test-slave-zpfg krb5kdc[6253](info): set up 2 sockets
Aug 09 00:36:35 dist-test-slave-zpfg krb5kdc[6253](info): commencing operation
krb5kdc: starting...
W20260809 00:36:36.660655 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:36 dist-test-slave-zpfg krb5kdc[6253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:36:36.798784 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45523
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:45523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:36.881662 6266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:36.881887 6266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:36.881911 6266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:36.883180 6266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:36.883226 6266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:36.883241 6266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:36.883255 6266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:36.883314 6266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:36.885186 6266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:45523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:36.885632 6266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:36.885983 6266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:36.891721 6272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:36.891731 6274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:36.891798 6266 server_base.cc:1034] running on GCE node
W20260809 00:36:36.891731 6271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:36.892129 6266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:36.892441 6266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:36.893569 6266 hybrid_clock.cc:648] HybridClock initialized: now 1786235796893550 us; error 40 us; skew 500 ppm
Aug 09 00:36:36 dist-test-slave-zpfg krb5kdc[6253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235796, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:36.896152 6266 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:36:36.896605 6266 webserver.cc:460] Webserver started at http://127.21.41.254:41573/ using document root <none> and password file <none>
I20260809 00:36:36.896862 6266 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:36.896935 6266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:36.897087 6266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:36.898161 6266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "ef5fccbb0ba44c5088435c0e0b548b02"
format_stamp: "Formatted at 2026-08-09 00:36:36 on dist-test-slave-zpfg"
server_key: "3e86a261863250b8545dd2578e8fd886"
server_key_iv: "9c3f4870d42e919187a64660db15868b"
server_key_version: "encryptionkey@0"
I20260809 00:36:36.898574 6266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "ef5fccbb0ba44c5088435c0e0b548b02"
format_stamp: "Formatted at 2026-08-09 00:36:36 on dist-test-slave-zpfg"
server_key: "3e86a261863250b8545dd2578e8fd886"
server_key_iv: "9c3f4870d42e919187a64660db15868b"
server_key_version: "encryptionkey@0"
I20260809 00:36:36.900012 6266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:36:36.900880 6281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:36.901098 6266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:36:36.901207 6266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "ef5fccbb0ba44c5088435c0e0b548b02"
format_stamp: "Formatted at 2026-08-09 00:36:36 on dist-test-slave-zpfg"
server_key: "3e86a261863250b8545dd2578e8fd886"
server_key_iv: "9c3f4870d42e919187a64660db15868b"
server_key_version: "encryptionkey@0"
I20260809 00:36:36.901285 6266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:36.942678 6266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:36.944931 6266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:36.945104 6266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:36.949374 6266 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:45523
I20260809 00:36:36.949405 6334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:45523 every 8 connection(s)
I20260809 00:36:36.949785 6266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:36.950703 6335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:36.953147 6335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02: Bootstrap starting.
I20260809 00:36:36.953758 6335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:36.954072 6335 log.cc:824] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:36.954531 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 6266
I20260809 00:36:36.954653 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:36.954864 21671 external_mini_cluster.cc:1428] Setting key 14ac884bac187a927e77f87da4a5f2ac
I20260809 00:36:36.955026 6335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02: No bootstrap required, opened a new log
I20260809 00:36:36.956961 6335 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef5fccbb0ba44c5088435c0e0b548b02" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45523 } }
I20260809 00:36:36.957123 6335 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:36.957170 6335 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef5fccbb0ba44c5088435c0e0b548b02, State: Initialized, Role: FOLLOWER
I20260809 00:36:36.957309 6335 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef5fccbb0ba44c5088435c0e0b548b02" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45523 } }
I20260809 00:36:36.957405 6335 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:36.957451 6335 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:36.957504 6335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:36.958031 6335 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef5fccbb0ba44c5088435c0e0b548b02" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45523 } }
I20260809 00:36:36.958151 6335 leader_election.cc:302] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef5fccbb0ba44c5088435c0e0b548b02; no voters:
I20260809 00:36:36.958333 6335 leader_election.cc:288] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:36.958390 6340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:36.958530 6340 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [term 1 LEADER]: Becoming Leader. State: Replica: ef5fccbb0ba44c5088435c0e0b548b02, State: Running, Role: LEADER
I20260809 00:36:36.958654 6335 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:36.958669 6340 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef5fccbb0ba44c5088435c0e0b548b02" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45523 } }
I20260809 00:36:36.959021 6341 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef5fccbb0ba44c5088435c0e0b548b02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef5fccbb0ba44c5088435c0e0b548b02" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45523 } } }
I20260809 00:36:36.959196 6341 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:36.959007 6340 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef5fccbb0ba44c5088435c0e0b548b02. Latest consensus state: current_term: 1 leader_uuid: "ef5fccbb0ba44c5088435c0e0b548b02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef5fccbb0ba44c5088435c0e0b548b02" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45523 } } }
I20260809 00:36:36.959506 6340 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:36.959663 6349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:36.960147 6349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:36.961805 6349 catalog_manager.cc:1347] Generated new cluster ID: cf9c300abceb46fd83ef2ecbc6ba4aa0
I20260809 00:36:36.961864 6349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:36:36 dist-test-slave-zpfg krb5kdc[6253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:36.967094 6339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:36.955448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53900 (local address 127.21.41.254:45523)
0809 00:36:36.955644 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:36.955654 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:36.955687 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:36:36.956287 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:36.956294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:36.956340 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:36.956500 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:36.957157 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:36.958615 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:36.959736 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:36.960510 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:36.960632 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:36.963508 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:36.963511 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:36.963512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:36.963523 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:36.965396 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:36.965823 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:36.965826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:36.965826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:36.965931 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:36.966177 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:36.966178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:36.966179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:36.966485 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:36.966628 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:36.966770 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 00:36:36.966900 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260809 00:36:36.969249 6349 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:36.969933 6349 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:36.987775 6349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02: Generated new TSK 0
I20260809 00:36:36.987999 6349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:36:37.021605 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45523
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:37.106660 6361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:37.106885 6361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:37.106926 6361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:37.108114 6361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:37.108189 6361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:37.108235 6361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:36:37.109700 6361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:37.109965 6361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:37.110204 6361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:37.115597 6367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:37.115677 6369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:37.115839 6366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:37.116209 6361 server_base.cc:1034] running on GCE node
I20260809 00:36:37.116464 6361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:37.116714 6361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:37.117915 6361 hybrid_clock.cc:648] HybridClock initialized: now 1786235797117905 us; error 30 us; skew 500 ppm
Aug 09 00:36:37 dist-test-slave-zpfg krb5kdc[6253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235797, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:37.120585 6361 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:36:37.121074 6361 webserver.cc:460] Webserver started at http://127.21.41.193:45333/ using document root <none> and password file <none>
I20260809 00:36:37.121331 6361 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:37.121412 6361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:37.121564 6361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:37.122689 6361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "79dd387c718440ff88636e365c3f08db"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "1970c109dfe867677a84cf7007457a5b"
server_key_iv: "5744cb1a4f38c9fc7d305f6abb4cdf0d"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.123056 6361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "79dd387c718440ff88636e365c3f08db"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "1970c109dfe867677a84cf7007457a5b"
server_key_iv: "5744cb1a4f38c9fc7d305f6abb4cdf0d"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.124430 6361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:37.125176 6376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.125352 6361 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:36:37.125456 6361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "79dd387c718440ff88636e365c3f08db"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "1970c109dfe867677a84cf7007457a5b"
server_key_iv: "5744cb1a4f38c9fc7d305f6abb4cdf0d"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.125540 6361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:37.140797 6361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:37.142896 6361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:37.143067 6361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:37.143317 6361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:37.143651 6361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:37.143711 6361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.143747 6361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:37.143802 6361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.149618 6361 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:34335
I20260809 00:36:37.149662 6489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:34335 every 8 connection(s)
I20260809 00:36:37.150012 6361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:36:37 dist-test-slave-zpfg krb5kdc[6253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235797, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:37.157691 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 6361
I20260809 00:36:37.157820 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:36:37.158056 21671 external_mini_cluster.cc:1428] Setting key 335aeb23f5c24d4d50aee55a2d6f5071
I20260809 00:36:37.159771 6339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.150829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:43797 (local address 127.21.41.254:45523)
0809 00:36:37.150903 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.150905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.151115 (+ 210us) server_negotiation.cc:407] Connection header received
0809 00:36:37.152049 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.152050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.152060 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.152120 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:37.152722 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.153223 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.154223 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.154521 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.154577 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.156630 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:37.156631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:37.156631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:37.156638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:37.158168 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:37.158672 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:37.158673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:37.158674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:37.158744 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:37.159051 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:37.159052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:37.159052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:37.159191 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:37.159267 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.159583 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.159675 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:36:37.159983 6493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.150814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45523 (local address 127.21.41.193:43797)
0809 00:36:37.150978 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:37.151021 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:37.151958 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:37.152181 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:37.152185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:37.152456 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:37.152647 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.152651 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.153332 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.153332 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.154145 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.154148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.154594 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.154595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.154702 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.155182 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:37.155191 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:37.156544 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:37.158261 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:37.158315 (+ 54us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:37.158318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:37.158584 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:37.158844 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:37.158844 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:37.158862 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:37.158965 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:37.159257 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:37.159264 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:37.159443 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 00:36:37.159563 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 00:36:37.159728 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260809 00:36:37.160457 6491 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45523
I20260809 00:36:37.160557 6491 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:37.160699 6491 heartbeater.cc:511] Master 127.21.41.254:45523 requested a full tablet report, sending...
I20260809 00:36:37.161259 6299 ts_manager.cc:194] Registered new tserver with Master: 79dd387c718440ff88636e365c3f08db (127.21.41.193:34335)
I20260809 00:36:37.162235 6299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:43797
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:36:37.186084 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45523
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:37.274513 6496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:37.274742 6496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:37.274768 6496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:37.275925 6496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:37.275964 6496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:37.275990 6496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:36:37.277379 6496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:37.277568 6496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:37.277755 6496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:37.282852 6501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:37.282900 6504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:37.283058 6502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:37.283110 6496 server_base.cc:1034] running on GCE node
I20260809 00:36:37.283326 6496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:37.283581 6496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:37.284761 6496 hybrid_clock.cc:648] HybridClock initialized: now 1786235797284756 us; error 39 us; skew 500 ppm
Aug 09 00:36:37 dist-test-slave-zpfg krb5kdc[6253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235797, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:37.287503 6496 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:36:37.287966 6496 webserver.cc:460] Webserver started at http://127.21.41.194:35923/ using document root <none> and password file <none>
I20260809 00:36:37.288156 6496 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:37.288206 6496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:37.288297 6496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:37.289348 6496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "700142a0ff0d404f8319cb46b1eadb1b"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "d65c85a8f23113683d8c94db1436ec69"
server_key_iv: "c3fcb7f4f90b69ddce21ea05f5c05be0"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.289659 6496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "700142a0ff0d404f8319cb46b1eadb1b"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "d65c85a8f23113683d8c94db1436ec69"
server_key_iv: "c3fcb7f4f90b69ddce21ea05f5c05be0"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.291147 6496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:36:37.291936 6511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.292135 6496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:36:37.292236 6496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "700142a0ff0d404f8319cb46b1eadb1b"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "d65c85a8f23113683d8c94db1436ec69"
server_key_iv: "c3fcb7f4f90b69ddce21ea05f5c05be0"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.292316 6496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:37.303730 6496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:37.305931 6496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:37.306046 6496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:37.306280 6496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:37.306632 6496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:37.306670 6496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.306694 6496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:37.306707 6496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.312891 6496 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:36437
I20260809 00:36:37.312938 6626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:36437 every 8 connection(s)
I20260809 00:36:37.313274 6496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:36:37 dist-test-slave-zpfg krb5kdc[6253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235797, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:37.321583 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 6496
I20260809 00:36:37.321686 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:36:37.321966 21671 external_mini_cluster.cc:1428] Setting key fc76af82d81b394217a6bef13e1cc643
I20260809 00:36:37.323772 6339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.314059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:56697 (local address 127.21.41.254:45523)
0809 00:36:37.314147 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.314150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.314397 (+ 247us) server_negotiation.cc:407] Connection header received
0809 00:36:37.314918 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.314919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.314931 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.314982 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:37.315966 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.316492 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.317581 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.317774 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.317834 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.320449 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:37.320450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:37.320451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:37.320458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:37.322162 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:37.322727 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:37.322728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:37.322729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:37.322796 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:37.323097 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:37.323098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:37.323098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:37.323224 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:37.323274 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.323623 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.323705 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 00:36:37.324060 6629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.314017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45523 (local address 127.21.41.194:56697)
0809 00:36:37.314224 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:37.314263 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:37.314789 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:37.315061 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:37.315064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:37.315677 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:37.315879 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.315884 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.316593 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.316594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.317481 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.317484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.317883 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.317884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.318001 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.318637 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:37.318648 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:37.320347 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:37.322318 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:37.322326 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:37.322328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:37.322655 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:37.322875 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:37.322876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:37.322876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:37.323025 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:37.323295 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:37.323301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:37.323478 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 00:36:37.323588 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 00:36:37.323776 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":118,"threads_started":1}
I20260809 00:36:37.324616 6627 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45523
I20260809 00:36:37.324724 6627 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:37.324910 6627 heartbeater.cc:511] Master 127.21.41.254:45523 requested a full tablet report, sending...
I20260809 00:36:37.325291 6299 ts_manager.cc:194] Registered new tserver with Master: 700142a0ff0d404f8319cb46b1eadb1b (127.21.41.194:36437)
I20260809 00:36:37.325814 6299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:56697
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:36:37.349627 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45523
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:37.431898 6632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:37.432153 6632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:37.432191 6632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:37.433358 6632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:37.433430 6632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:37.433475 6632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:36:37.434964 6632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:45523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:37.435240 6632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:37.435482 6632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:37.440575 6640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:37.440575 6637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:37.440687 6632 server_base.cc:1034] running on GCE node
W20260809 00:36:37.440582 6638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:37.441146 6632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:37.441416 6632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:37.442586 6632 hybrid_clock.cc:648] HybridClock initialized: now 1786235797442583 us; error 42 us; skew 500 ppm
Aug 09 00:36:37 dist-test-slave-zpfg krb5kdc[6253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235797, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:37.445585 6632 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:36:37.446059 6632 webserver.cc:460] Webserver started at http://127.21.41.195:34781/ using document root <none> and password file <none>
I20260809 00:36:37.446302 6632 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:37.446398 6632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:37.446539 6632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:37.447600 6632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "90cc93e5b6964c3a81b45fe7df52f8c7"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "bafc523ddfb327d7ed138b60fae8ca86"
server_key_iv: "d6896007e7f9a9eee7d9d4a8f5adcbf6"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.447996 6632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "90cc93e5b6964c3a81b45fe7df52f8c7"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "bafc523ddfb327d7ed138b60fae8ca86"
server_key_iv: "d6896007e7f9a9eee7d9d4a8f5adcbf6"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.449810 6632 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 00:36:37.450686 6647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.450989 6632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:36:37.451103 6632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "90cc93e5b6964c3a81b45fe7df52f8c7"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "bafc523ddfb327d7ed138b60fae8ca86"
server_key_iv: "d6896007e7f9a9eee7d9d4a8f5adcbf6"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.451190 6632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:37.469570 6632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:37.471663 6632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:37.471868 6632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:37.472138 6632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:37.472476 6632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:37.472535 6632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.472579 6632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:37.472604 6632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.478840 6632 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:36201
I20260809 00:36:37.478883 6762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:36201 every 8 connection(s)
I20260809 00:36:37.479296 6632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:36:37.485239 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 6632
I20260809 00:36:37.485363 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:36:37.485544 21671 external_mini_cluster.cc:1428] Setting key 90d67817f5990dfdc739a14ad0c2e0ac
Aug 09 00:36:37 dist-test-slave-zpfg krb5kdc[6253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235797, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:37.489876 6339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.480096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:33409 (local address 127.21.41.254:45523)
0809 00:36:37.480184 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.480187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.480384 (+ 197us) server_negotiation.cc:407] Connection header received
0809 00:36:37.480928 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.480929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.480951 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.481030 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:37.482067 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.482679 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.483728 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.483930 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.484029 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.486797 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:37.486798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:37.486799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:37.486807 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:37.488408 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:37.488860 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:37.488861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:37.488862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:37.488924 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:37.489208 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:37.489209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:37.489209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:37.489328 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:37.489404 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.489700 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.489784 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:36:37.490063 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.480010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45523 (local address 127.21.41.195:33409)
0809 00:36:37.480246 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:37.480294 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:37.480811 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:37.481113 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:37.481117 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:37.481743 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:37.481949 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.481953 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.482791 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.482792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.483612 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.483614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.484038 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.484039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.484182 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.484688 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:37.484698 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:37.486627 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:37.488514 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:37.488523 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:37.488525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:37.488764 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:37.488994 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:37.488995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:37.488995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:37.489115 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:37.489406 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:37.489413 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:37.489554 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 00:36:37.489660 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 00:36:37.489823 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":92,"threads_started":1}
I20260809 00:36:37.490561 6763 heartbeater.cc:347] Connected to a master server at 127.21.41.254:45523
I20260809 00:36:37.490661 6763 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:37.490823 6763 heartbeater.cc:511] Master 127.21.41.254:45523 requested a full tablet report, sending...
I20260809 00:36:37.491176 6299 ts_manager.cc:194] Registered new tserver with Master: 90cc93e5b6964c3a81b45fe7df52f8c7 (127.21.41.195:36201)
I20260809 00:36:37.491523 6299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:33409
I20260809 00:36:37.497788 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:36:37.505239 6339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.499226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.21.41.254:45523)
0809 00:36:37.499279 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.499281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.499382 (+ 101us) server_negotiation.cc:407] Connection header received
0809 00:36:37.499420 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.499421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.499430 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.499479 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:37.500087 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.500565 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.501327 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.501502 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.501551 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.502934 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:37.502935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:37.502935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:37.502941 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:37.504188 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:37.504528 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:37.504529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:37.504529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:37.504580 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:37.504744 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:37.504745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:37.504745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:37.504934 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:37.505012 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.505115 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.505176 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:36:37.506712 6299 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:36:37.508236 6299 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:36:37.514963 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.510903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:34335 (local address 127.0.0.1:43076)
0809 00:36:37.511180 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:37.511190 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:37.511231 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:37.511788 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:37.511789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:37.511791 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:37.511845 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.511854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.512811 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.512812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.513866 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.513869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.514644 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.514645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.514719 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.514733 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:36:37.514807 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:36:37.514860 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":45,"thread_start_us":66,"threads_started":1}
I20260809 00:36:37.514973 6778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.511031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.21.41.193:34335)
0809 00:36:37.511431 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.511437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.511448 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:36:37.511491 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.511494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.511508 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.511796 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:37.511923 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.512692 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.513957 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.514546 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.514644 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.514711 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.514837 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.514885 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":107,"threads_started":1}
I20260809 00:36:37.515322 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.510671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:36437 (local address 127.0.0.1:43768)
0809 00:36:37.510794 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:37.510835 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:37.510924 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:37.511232 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:37.511235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:37.511237 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:37.511322 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.511325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.512736 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.512737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.514010 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.514012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.515047 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.515048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.515120 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.515140 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 00:36:37.515204 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 00:36:37.515254 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":26,"thread_start_us":51,"threads_started":1}
I20260809 00:36:37.515370 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.510835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43768 (local address 127.21.41.194:36437)
0809 00:36:37.511048 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.511054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.511065 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:36:37.511113 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.511116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.511133 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.511223 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:37.511494 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.512617 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.514103 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.514898 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.514959 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.515011 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.515237 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.515280 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":89,"threads_started":1}
I20260809 00:36:37.516179 6560 tablet_service.cc:1467] Processing CreateTablet for tablet eba7ac03e7ca4133900b28a0ddaf1afb (DEFAULT_TABLE table=test-table [id=a637b75f02204f94b9108e50529e86ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:37.516177 6424 tablet_service.cc:1467] Processing CreateTablet for tablet eba7ac03e7ca4133900b28a0ddaf1afb (DEFAULT_TABLE table=test-table [id=a637b75f02204f94b9108e50529e86ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:37.516561 6560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eba7ac03e7ca4133900b28a0ddaf1afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:37.516561 6424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eba7ac03e7ca4133900b28a0ddaf1afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:37.518594 6779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.511234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46226 (local address 127.21.41.195:36201)
0809 00:36:37.511449 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.511453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.511708 (+ 255us) server_negotiation.cc:407] Connection header received
0809 00:36:37.511755 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.511757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.511774 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.512002 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:37.512112 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.513276 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.514615 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.515382 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.515458 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.515514 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.518443 (+ 2929us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.518505 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
I20260809 00:36:37.518860 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.511065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:36201 (local address 127.0.0.1:46226)
0809 00:36:37.511619 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:37.511629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:37.511680 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:37.512017 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:37.512018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:37.512019 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:37.512051 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.512052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.513392 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.513393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.514512 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.514514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.515578 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.515579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.515659 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.515664 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:36:37.515717 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:36:37.518812 (+ 3095us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":66,"threads_started":1}
I20260809 00:36:37.519543 6695 tablet_service.cc:1467] Processing CreateTablet for tablet eba7ac03e7ca4133900b28a0ddaf1afb (DEFAULT_TABLE table=test-table [id=a637b75f02204f94b9108e50529e86ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:37.519924 6695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eba7ac03e7ca4133900b28a0ddaf1afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:37.520017 6780 tablet_bootstrap.cc:492] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db: Bootstrap starting.
I20260809 00:36:37.520773 6780 tablet_bootstrap.cc:654] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:37.521396 6780 log.cc:824] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:37.521754 6781 tablet_bootstrap.cc:492] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: Bootstrap starting.
I20260809 00:36:37.522204 6780 tablet_bootstrap.cc:492] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db: No bootstrap required, opened a new log
I20260809 00:36:37.522251 6780 ts_tablet_manager.cc:1397] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:36:37.522415 6781 tablet_bootstrap.cc:654] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:37.522993 6781 log.cc:824] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:37.523792 6781 tablet_bootstrap.cc:492] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: No bootstrap required, opened a new log
I20260809 00:36:37.523968 6784 tablet_bootstrap.cc:492] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7: Bootstrap starting.
I20260809 00:36:37.524004 6781 ts_tablet_manager.cc:1397] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:36:37.524228 6780 raft_consensus.cc:348] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700142a0ff0d404f8319cb46b1eadb1b" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36437 } } peers { permanent_uuid: "79dd387c718440ff88636e365c3f08db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34335 } } peers { permanent_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 36201 } }
I20260809 00:36:37.524438 6780 raft_consensus.cc:374] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:37.524475 6780 raft_consensus.cc:729] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79dd387c718440ff88636e365c3f08db, State: Initialized, Role: FOLLOWER
I20260809 00:36:37.524626 6784 tablet_bootstrap.cc:654] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:37.524626 6780 consensus_queue.cc:260] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700142a0ff0d404f8319cb46b1eadb1b" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36437 } } peers { permanent_uuid: "79dd387c718440ff88636e365c3f08db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34335 } } peers { permanent_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 36201 } }
I20260809 00:36:37.524923 6780 ts_tablet_manager.cc:1428] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:36:37.524932 6491 heartbeater.cc:503] Master 127.21.41.254:45523 was elected leader, sending a full tablet report...
I20260809 00:36:37.525221 6784 log.cc:824] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:37.525949 6781 raft_consensus.cc:348] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700142a0ff0d404f8319cb46b1eadb1b" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36437 } } peers { permanent_uuid: "79dd387c718440ff88636e365c3f08db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34335 } } peers { permanent_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 36201 } }
I20260809 00:36:37.526130 6781 raft_consensus.cc:374] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:37.526206 6781 raft_consensus.cc:729] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 700142a0ff0d404f8319cb46b1eadb1b, State: Initialized, Role: FOLLOWER
I20260809 00:36:37.526381 6781 consensus_queue.cc:260] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700142a0ff0d404f8319cb46b1eadb1b" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36437 } } peers { permanent_uuid: "79dd387c718440ff88636e365c3f08db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34335 } } peers { permanent_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 36201 } }
I20260809 00:36:37.526678 6781 ts_tablet_manager.cc:1428] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:36:37.526688 6627 heartbeater.cc:503] Master 127.21.41.254:45523 was elected leader, sending a full tablet report...
I20260809 00:36:37.526954 6784 tablet_bootstrap.cc:492] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7: No bootstrap required, opened a new log
I20260809 00:36:37.527029 6784 ts_tablet_manager.cc:1397] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:36:37.528446 6784 raft_consensus.cc:348] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700142a0ff0d404f8319cb46b1eadb1b" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36437 } } peers { permanent_uuid: "79dd387c718440ff88636e365c3f08db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34335 } } peers { permanent_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 36201 } }
I20260809 00:36:37.528600 6784 raft_consensus.cc:374] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:37.528647 6784 raft_consensus.cc:729] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90cc93e5b6964c3a81b45fe7df52f8c7, State: Initialized, Role: FOLLOWER
I20260809 00:36:37.528756 6784 consensus_queue.cc:260] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700142a0ff0d404f8319cb46b1eadb1b" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36437 } } peers { permanent_uuid: "79dd387c718440ff88636e365c3f08db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34335 } } peers { permanent_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 36201 } }
I20260809 00:36:37.529040 6784 ts_tablet_manager.cc:1428] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:36:37.529055 6763 heartbeater.cc:503] Master 127.21.41.254:45523 was elected leader, sending a full tablet report...
W20260809 00:36:37.563910 6628 tablet.cc:2369] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:36:37.616880 6787 raft_consensus.cc:482] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:36:37.617028 6787 raft_consensus.cc:504] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700142a0ff0d404f8319cb46b1eadb1b" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36437 } } peers { permanent_uuid: "79dd387c718440ff88636e365c3f08db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34335 } } peers { permanent_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 36201 } }
I20260809 00:36:37.617389 6787 leader_election.cc:288] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79dd387c718440ff88636e365c3f08db (127.21.41.193:34335), 90cc93e5b6964c3a81b45fe7df52f8c7 (127.21.41.195:36201)
I20260809 00:36:37.620697 6779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.617872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:44139 (local address 127.21.41.195:36201)
0809 00:36:37.617950 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.617952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.618073 (+ 121us) server_negotiation.cc:407] Connection header received
0809 00:36:37.618104 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.618105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.618116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.618185 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:37.618413 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.618881 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.619920 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.620371 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.620445 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.620454 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.620590 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.620625 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:36:37.620697 6778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.617696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:47379 (local address 127.21.41.193:34335)
0809 00:36:37.617770 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.617773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.617912 (+ 139us) server_negotiation.cc:407] Connection header received
0809 00:36:37.617950 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.617952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.617964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.618032 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:37.618253 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.618910 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.619875 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.620309 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.620393 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.620403 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.620576 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.620623 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:36:37.620724 6629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.617640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:34335 (local address 127.21.41.194:47379)
0809 00:36:37.617710 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:37.617742 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:37.617818 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:37.618068 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:37.618069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:37.618072 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:37.618162 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.618164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.619032 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.619033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.619772 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.619774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.620397 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.620397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.620482 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.620486 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:36:37.620546 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 00:36:37.620597 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 00:36:37.620731 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.617770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:36201 (local address 127.21.41.194:44139)
0809 00:36:37.617994 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:37.618004 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:37.618058 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:37.618218 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:37.618219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:37.618221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:37.618302 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.618304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.618980 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.618981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.619788 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.619790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.620447 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.620448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.620515 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.620517 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:36:37.620570 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:36:37.620598 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":21,"thread_start_us":147,"threads_started":1}
I20260809 00:36:37.621132 6444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eba7ac03e7ca4133900b28a0ddaf1afb" candidate_uuid: "700142a0ff0d404f8319cb46b1eadb1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79dd387c718440ff88636e365c3f08db" is_pre_election: true
I20260809 00:36:37.621157 6715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eba7ac03e7ca4133900b28a0ddaf1afb" candidate_uuid: "700142a0ff0d404f8319cb46b1eadb1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" is_pre_election: true
I20260809 00:36:37.621302 6444 raft_consensus.cc:2454] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 700142a0ff0d404f8319cb46b1eadb1b in term 0.
I20260809 00:36:37.621318 6715 raft_consensus.cc:2454] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 700142a0ff0d404f8319cb46b1eadb1b in term 0.
I20260809 00:36:37.621541 6512 leader_election.cc:302] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 700142a0ff0d404f8319cb46b1eadb1b, 79dd387c718440ff88636e365c3f08db; no voters:
I20260809 00:36:37.621687 6787 raft_consensus.cc:2781] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:36:37.621780 6787 raft_consensus.cc:482] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:36:37.621819 6787 raft_consensus.cc:3037] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:37.622578 6787 raft_consensus.cc:504] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700142a0ff0d404f8319cb46b1eadb1b" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36437 } } peers { permanent_uuid: "79dd387c718440ff88636e365c3f08db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34335 } } peers { permanent_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 36201 } }
I20260809 00:36:37.622736 6787 leader_election.cc:288] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [CANDIDATE]: Term 1 election: Requested vote from peers 79dd387c718440ff88636e365c3f08db (127.21.41.193:34335), 90cc93e5b6964c3a81b45fe7df52f8c7 (127.21.41.195:36201)
I20260809 00:36:37.622922 6444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eba7ac03e7ca4133900b28a0ddaf1afb" candidate_uuid: "700142a0ff0d404f8319cb46b1eadb1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79dd387c718440ff88636e365c3f08db"
I20260809 00:36:37.622918 6715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eba7ac03e7ca4133900b28a0ddaf1afb" candidate_uuid: "700142a0ff0d404f8319cb46b1eadb1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7"
I20260809 00:36:37.622990 6444 raft_consensus.cc:3037] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:37.623018 6715 raft_consensus.cc:3037] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:37.623704 6444 raft_consensus.cc:2454] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 700142a0ff0d404f8319cb46b1eadb1b in term 1.
I20260809 00:36:37.623775 6715 raft_consensus.cc:2454] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 700142a0ff0d404f8319cb46b1eadb1b in term 1.
I20260809 00:36:37.623952 6512 leader_election.cc:302] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 700142a0ff0d404f8319cb46b1eadb1b, 90cc93e5b6964c3a81b45fe7df52f8c7; no voters:
I20260809 00:36:37.624053 6787 raft_consensus.cc:2781] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:37.624223 6787 raft_consensus.cc:686] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 1 LEADER]: Becoming Leader. State: Replica: 700142a0ff0d404f8319cb46b1eadb1b, State: Running, Role: LEADER
I20260809 00:36:37.624341 6787 consensus_queue.cc:237] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700142a0ff0d404f8319cb46b1eadb1b" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36437 } } peers { permanent_uuid: "79dd387c718440ff88636e365c3f08db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34335 } } peers { permanent_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 36201 } }
I20260809 00:36:37.625291 6299 catalog_manager.cc:5495] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 700142a0ff0d404f8319cb46b1eadb1b (127.21.41.194). New cstate: current_term: 1 leader_uuid: "700142a0ff0d404f8319cb46b1eadb1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700142a0ff0d404f8319cb46b1eadb1b" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 36437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79dd387c718440ff88636e365c3f08db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 36201 } health_report { overall_health: UNKNOWN } } }
I20260809 00:36:37.630499 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.628026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43784 (local address 127.21.41.194:36437)
0809 00:36:37.628067 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.628070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.628098 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:36:37.628183 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.628184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.628201 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.628269 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:36:37.628416 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.628888 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.629682 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.629895 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.629973 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.630140 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:36:37.630236 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:36:37.630338 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.630386 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.630405 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:36:37.632213 6715 raft_consensus.cc:1261] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7 [term 1 FOLLOWER]: Refusing update from remote peer 700142a0ff0d404f8319cb46b1eadb1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:37.632215 6444 raft_consensus.cc:1261] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db [term 1 FOLLOWER]: Refusing update from remote peer 700142a0ff0d404f8319cb46b1eadb1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:37.632550 6790 consensus_queue.cc:1035] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "79dd387c718440ff88636e365c3f08db" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:37.632660 6787 consensus_queue.cc:1035] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90cc93e5b6964c3a81b45fe7df52f8c7" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 36201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:37.635123 6794 mvcc.cc:205] Tried to move back new op lower bound from 7316421827098898432 to 7316421827070115840. Current Snapshot: MvccSnapshot[applied={T|T < 7316421827098898432}]
I20260809 00:36:37.635247 6795 mvcc.cc:205] Tried to move back new op lower bound from 7316421827098898432 to 7316421827070115840. Current Snapshot: MvccSnapshot[applied={T|T < 7316421827098898432}]
I20260809 00:36:37.637080 6299 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:37.637164 6299 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:37.637887 6299 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef5fccbb0ba44c5088435c0e0b548b02: Sending DeleteTablet for 3 replicas of tablet eba7ac03e7ca4133900b28a0ddaf1afb
I20260809 00:36:37.638250 6560 tablet_service.cc:1514] Processing DeleteTablet for tablet eba7ac03e7ca4133900b28a0ddaf1afb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:37 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:43768
I20260809 00:36:37.638298 6695 tablet_service.cc:1514] Processing DeleteTablet for tablet eba7ac03e7ca4133900b28a0ddaf1afb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:37 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:46226
I20260809 00:36:37.638536 6803 tablet_replica.cc:330] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: stopping tablet replica
I20260809 00:36:37.638510 6424 tablet_service.cc:1514] Processing DeleteTablet for tablet eba7ac03e7ca4133900b28a0ddaf1afb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:37 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:43076
I20260809 00:36:37.638639 6803 raft_consensus.cc:2229] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:36:37.638736 6803 raft_consensus.cc:2258] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:37.638978 6805 tablet_replica.cc:330] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db: stopping tablet replica
I20260809 00:36:37.639081 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 6361
I20260809 00:36:37.639098 6805 raft_consensus.cc:2229] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:37.639186 6805 raft_consensus.cc:2258] T eba7ac03e7ca4133900b28a0ddaf1afb P 79dd387c718440ff88636e365c3f08db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:37.639235 6803 ts_tablet_manager.cc:1905] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:36:37.639652 6804 tablet_replica.cc:330] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7: stopping tablet replica
I20260809 00:36:37.639731 6804 raft_consensus.cc:2229] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:37.639813 6804 raft_consensus.cc:2258] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:37.641239 6803 ts_tablet_manager.cc:1918] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:37.641290 6803 log.cc:1196] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/eba7ac03e7ca4133900b28a0ddaf1afb
I20260809 00:36:37.641484 6803 ts_tablet_manager.cc:1939] T eba7ac03e7ca4133900b28a0ddaf1afb P 700142a0ff0d404f8319cb46b1eadb1b: Deleting consensus metadata
I20260809 00:36:37.641809 6282 catalog_manager.cc:4841] TS 700142a0ff0d404f8319cb46b1eadb1b (127.21.41.194:36437): tablet eba7ac03e7ca4133900b28a0ddaf1afb (table test-table [id=a637b75f02204f94b9108e50529e86ce]) successfully deleted
W20260809 00:36:37.646380 6282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:34335 (error 108)
I20260809 00:36:37.646600 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 6496
I20260809 00:36:37.647574 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.646758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:34335 (local address 127.0.0.1:43082)
0809 00:36:37.647497 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:37.647542 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:34335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":715}
I20260809 00:36:37.647588 6804 ts_tablet_manager.cc:1905] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:36:37.647722 6282 catalog_manager.cc:4584] TS 79dd387c718440ff88636e365c3f08db (127.21.41.193:34335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eba7ac03e7ca4133900b28a0ddaf1afb: Network error: Client connection negotiation failed: client connection to 127.21.41.193:34335: connect: Connection refused (error 111)
I20260809 00:36:37.649477 6804 ts_tablet_manager.cc:1918] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:37.649550 6804 log.cc:1196] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/eba7ac03e7ca4133900b28a0ddaf1afb
I20260809 00:36:37.649879 6804 ts_tablet_manager.cc:1939] T eba7ac03e7ca4133900b28a0ddaf1afb P 90cc93e5b6964c3a81b45fe7df52f8c7: Deleting consensus metadata
I20260809 00:36:37.650453 6282 catalog_manager.cc:4841] TS 90cc93e5b6964c3a81b45fe7df52f8c7 (127.21.41.195:36201): tablet eba7ac03e7ca4133900b28a0ddaf1afb (table test-table [id=a637b75f02204f94b9108e50529e86ce]) successfully deleted
I20260809 00:36:37.652817 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 6632
I20260809 00:36:37.657580 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 6266
2026-08-09T00:36:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2040 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T00:36:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:37Z Disabled control of system clock
I20260809 00:36:37.686044 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:41249
--webserver_interface=127.21.41.254
--webserver_port=0
--builtin_ntp_servers=127.21.41.212:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:41249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:36:37.767481 6810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:37.767707 6810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:37.767728 6810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:37.768831 6810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:37.768867 6810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:37.768879 6810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:37.768893 6810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:37.768904 6810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:37.770258 6810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:41249
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:41249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:37.770521 6810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:37.770784 6810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:37.775650 6816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:37.775619 6815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:37.775658 6818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:37.775910 6810 server_base.cc:1034] running on GCE node
I20260809 00:36:37.776146 6810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:37.776443 6810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:37.777612 6810 hybrid_clock.cc:648] HybridClock initialized: now 1786235797777588 us; error 37 us; skew 500 ppm
I20260809 00:36:37.778934 6810 webserver.cc:460] Webserver started at http://127.21.41.254:45661/ using document root <none> and password file <none>
I20260809 00:36:37.779179 6810 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:37.779249 6810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:37.779438 6810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:37.780522 6810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "9f098b240b6649b2a85b5ec8ae7e5c53"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "15880ad24e899bc4a76530645030ad50"
server_key_iv: "5c3f9804b960ec70f5fab8f37bc3f843"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.780894 6810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "9f098b240b6649b2a85b5ec8ae7e5c53"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "15880ad24e899bc4a76530645030ad50"
server_key_iv: "5c3f9804b960ec70f5fab8f37bc3f843"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.782227 6810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:36:37.783020 6824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.783198 6810 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:36:37.783314 6810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "9f098b240b6649b2a85b5ec8ae7e5c53"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "15880ad24e899bc4a76530645030ad50"
server_key_iv: "5c3f9804b960ec70f5fab8f37bc3f843"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.783409 6810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:37.799944 6810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:37.800308 6810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:37.800477 6810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:37.804574 6810 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:41249
I20260809 00:36:37.804626 6876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:41249 every 8 connection(s)
I20260809 00:36:37.805001 6810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:37.806022 6877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:37.808315 6877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53: Bootstrap starting.
I20260809 00:36:37.808857 6877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:37.809099 6877 log.cc:824] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:37.809736 6877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53: No bootstrap required, opened a new log
I20260809 00:36:37.810583 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 6810
I20260809 00:36:37.810685 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:37.810881 21671 external_mini_cluster.cc:1428] Setting key 3fa220f864a3b1ee8d4f1a4e7a1a877a
I20260809 00:36:37.811913 6877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f098b240b6649b2a85b5ec8ae7e5c53" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41249 } }
I20260809 00:36:37.812084 6877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:37.812124 6877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f098b240b6649b2a85b5ec8ae7e5c53, State: Initialized, Role: FOLLOWER
I20260809 00:36:37.812311 6877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f098b240b6649b2a85b5ec8ae7e5c53" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41249 } }
I20260809 00:36:37.812397 6877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:37.812420 6877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:37.812492 6877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:37.813035 6877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f098b240b6649b2a85b5ec8ae7e5c53" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41249 } }
I20260809 00:36:37.813122 6877 leader_election.cc:302] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f098b240b6649b2a85b5ec8ae7e5c53; no voters:
I20260809 00:36:37.813357 6877 leader_election.cc:288] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:37.813400 6883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:37.813557 6883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [term 1 LEADER]: Becoming Leader. State: Replica: 9f098b240b6649b2a85b5ec8ae7e5c53, State: Running, Role: LEADER
I20260809 00:36:37.813624 6877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:37.813741 6883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f098b240b6649b2a85b5ec8ae7e5c53" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41249 } }
I20260809 00:36:37.814116 6883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f098b240b6649b2a85b5ec8ae7e5c53. Latest consensus state: current_term: 1 leader_uuid: "9f098b240b6649b2a85b5ec8ae7e5c53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f098b240b6649b2a85b5ec8ae7e5c53" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41249 } } }
I20260809 00:36:37.814429 6883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:37.814306 6885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f098b240b6649b2a85b5ec8ae7e5c53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f098b240b6649b2a85b5ec8ae7e5c53" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41249 } } }
I20260809 00:36:37.814680 6885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:37.815003 6892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:37.815907 6892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:37.817436 6882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.811405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.21.41.254:41249)
0809 00:36:37.811623 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.811630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.811655 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:36:37.811886 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.811892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.811947 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.812054 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:37.812343 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.813989 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.815195 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.815730 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.815883 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.816616 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:37.816619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:37.816620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:37.816871 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:37.816905 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:37.816918 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:37.817022 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.817129 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.817209 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260809 00:36:37.818153 6892 catalog_manager.cc:1347] Generated new cluster ID: 866b1b7b6a0d47a0bf457845ac0bbd80
I20260809 00:36:37.818209 6892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:37.826297 6892 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:37.826936 6892 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:37.834187 6892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53: Generated new TSK 0
I20260809 00:36:37.834405 6892 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:37.836031 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:41249
--builtin_ntp_servers=127.21.41.212:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:36:37.920683 6902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:37.921090 6902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:37.921134 6902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:37.922315 6902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:37.922405 6902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:37.922477 6902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:36:37.923892 6902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:41249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:37.924142 6902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:37.924477 6902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:37.929836 6910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:37.929814 6907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:37.930047 6902 server_base.cc:1034] running on GCE node
W20260809 00:36:37.929808 6908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:37.930397 6902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:37.930692 6902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:37.931859 6902 hybrid_clock.cc:648] HybridClock initialized: now 1786235797931846 us; error 31 us; skew 500 ppm
I20260809 00:36:37.933300 6902 webserver.cc:460] Webserver started at http://127.21.41.193:45889/ using document root <none> and password file <none>
I20260809 00:36:37.933552 6902 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:37.933626 6902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:37.933810 6902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:37.934988 6902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "0a184e31bbc648229b1c6dd28c3bd390"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "371814a44f9563e2aba7760aba3ed1fd"
server_key_iv: "70ed85dacaaf97e1668bd342d85c6bea"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.935369 6902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "0a184e31bbc648229b1c6dd28c3bd390"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "371814a44f9563e2aba7760aba3ed1fd"
server_key_iv: "70ed85dacaaf97e1668bd342d85c6bea"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.936759 6902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:36:37.937625 6916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.937840 6902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:37.937956 6902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "0a184e31bbc648229b1c6dd28c3bd390"
format_stamp: "Formatted at 2026-08-09 00:36:37 on dist-test-slave-zpfg"
server_key: "371814a44f9563e2aba7760aba3ed1fd"
server_key_iv: "70ed85dacaaf97e1668bd342d85c6bea"
server_key_version: "encryptionkey@0"
I20260809 00:36:37.938042 6902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:37.952419 6902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:37.952800 6902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:37.952975 6902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:37.953229 6902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:37.953579 6902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:37.953636 6902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.953682 6902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:37.953706 6902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:37.959861 6902 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:44857
I20260809 00:36:37.959934 7029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:44857 every 8 connection(s)
I20260809 00:36:37.960300 6902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:36:37.961762 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 6902
I20260809 00:36:37.961884 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:36:37.962137 21671 external_mini_cluster.cc:1428] Setting key 1d323e8e65bf49c8818d5c209014fbd7
I20260809 00:36:37.963321 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:41249
--builtin_ntp_servers=127.21.41.212:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 00:36:37.966676 6882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.961098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:51543 (local address 127.21.41.254:41249)
0809 00:36:37.961176 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:37.961178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:37.961434 (+ 256us) server_negotiation.cc:407] Connection header received
0809 00:36:37.962015 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:37.962017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:37.962036 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:37.962116 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:37.963090 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.963840 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.965021 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.965212 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.965268 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.966330 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:37.966331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:37.966331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:37.966335 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:37.966405 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:37.966410 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:37.966465 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:37.966583 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 00:36:37.966616 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:36:37.966970 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:37.961086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:41249 (local address 127.21.41.193:51543)
0809 00:36:37.961296 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:37.961333 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:37.961906 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:37.962176 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:37.962179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:37.962188 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:37.962992 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.963000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.963936 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.963937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.964927 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:37.964930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:37.965305 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:37.965307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:37.965501 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:37.966168 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:36:37.966169 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:37.966208 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:36:37.966237 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:37.966486 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:37.966493 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:37.966496 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:36:37.966578 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 00:36:37.966700 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260809 00:36:37.967572 7030 heartbeater.cc:347] Connected to a master server at 127.21.41.254:41249
I20260809 00:36:37.967689 7030 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:37.967918 7030 heartbeater.cc:511] Master 127.21.41.254:41249 requested a full tablet report, sending...
I20260809 00:36:37.968474 6841 ts_manager.cc:194] Registered new tserver with Master: 0a184e31bbc648229b1c6dd28c3bd390 (127.21.41.193:44857)
I20260809 00:36:37.969089 6841 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.193:51543
W20260809 00:36:38.050295 7033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:38.050582 7033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:38.050606 7033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:38.051810 7033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:38.051857 7033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:38.051883 7033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:36:38.053244 7033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:41249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:38.053491 7033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:38.053825 7033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:38.059058 7039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:38.059067 7038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:38.059089 7033 server_base.cc:1034] running on GCE node
W20260809 00:36:38.059058 7041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:38.059619 7033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:38.059883 7033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:38.061060 7033 hybrid_clock.cc:648] HybridClock initialized: now 1786235798061035 us; error 37 us; skew 500 ppm
I20260809 00:36:38.066637 7033 webserver.cc:460] Webserver started at http://127.21.41.194:35773/ using document root <none> and password file <none>
I20260809 00:36:38.066926 7033 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:38.067008 7033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:38.067167 7033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:38.068325 7033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "c8f4b042ca444d7e92ca38dcad571df0"
format_stamp: "Formatted at 2026-08-09 00:36:38 on dist-test-slave-zpfg"
server_key: "0101dec7d6b5f699ef29fc37ea8dd5ea"
server_key_iv: "37d5c453fe95524758a8aa928d62bd0f"
server_key_version: "encryptionkey@0"
I20260809 00:36:38.068706 7033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "c8f4b042ca444d7e92ca38dcad571df0"
format_stamp: "Formatted at 2026-08-09 00:36:38 on dist-test-slave-zpfg"
server_key: "0101dec7d6b5f699ef29fc37ea8dd5ea"
server_key_iv: "37d5c453fe95524758a8aa928d62bd0f"
server_key_version: "encryptionkey@0"
I20260809 00:36:38.070088 7033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:36:38.070935 7047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:38.071133 7033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:36:38.071233 7033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "c8f4b042ca444d7e92ca38dcad571df0"
format_stamp: "Formatted at 2026-08-09 00:36:38 on dist-test-slave-zpfg"
server_key: "0101dec7d6b5f699ef29fc37ea8dd5ea"
server_key_iv: "37d5c453fe95524758a8aa928d62bd0f"
server_key_version: "encryptionkey@0"
I20260809 00:36:38.071288 7033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:38.086149 7033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:38.086531 7033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:38.086647 7033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:38.086889 7033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:38.087168 7033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:38.087198 7033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:38.087220 7033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:38.087234 7033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:38.092936 7033 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:46447
I20260809 00:36:38.093022 7160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:46447 every 8 connection(s)
I20260809 00:36:38.093407 7033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:36:38.098878 6882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.094222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:37993 (local address 127.21.41.254:41249)
0809 00:36:38.094326 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:38.094328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:38.094561 (+ 233us) server_negotiation.cc:407] Connection header received
0809 00:36:38.095137 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:38.095138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:38.095150 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:38.095197 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:38.095921 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.096427 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.097428 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.097630 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.097706 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.098574 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:38.098576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:38.098576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:38.098581 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:38.098597 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:38.098601 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:38.098672 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:38.098764 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 00:36:38.098796 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:36:38.099126 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.094214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:41249 (local address 127.21.41.194:37993)
0809 00:36:38.094437 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:38.094472 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:38.095015 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:38.095266 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:38.095271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:38.095280 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:38.095810 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.095814 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.096527 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.096528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.097319 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.097321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.097725 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.097726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.097863 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.098434 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:36:38.098434 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:38.098464 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:36:38.098486 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:38.098672 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:38.098678 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:38.098681 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:36:38.098795 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 00:36:38.098898 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
I20260809 00:36:38.099500 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 7033
I20260809 00:36:38.099604 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:36:38.099694 7161 heartbeater.cc:347] Connected to a master server at 127.21.41.254:41249
I20260809 00:36:38.099820 7161 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:38.099841 21671 external_mini_cluster.cc:1428] Setting key 2b2bf4edfc9fdcb3c503d61dc0a7ffc0
I20260809 00:36:38.100013 7161 heartbeater.cc:511] Master 127.21.41.254:41249 requested a full tablet report, sending...
I20260809 00:36:38.100432 6840 ts_manager.cc:194] Registered new tserver with Master: c8f4b042ca444d7e92ca38dcad571df0 (127.21.41.194:46447)
I20260809 00:36:38.100966 6840 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.194:37993
I20260809 00:36:38.100981 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:41249
--builtin_ntp_servers=127.21.41.212:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:36:38.185403 7164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:38.185696 7164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:38.185739 7164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:38.187045 7164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:38.187107 7164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:38.187150 7164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:36:38.188531 7164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:41249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:38.188834 7164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:38.189155 7164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:38.194512 7169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:38.194520 7170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:38.194516 7172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:38.194532 7164 server_base.cc:1034] running on GCE node
I20260809 00:36:38.195029 7164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:38.195282 7164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:38.196447 7164 hybrid_clock.cc:648] HybridClock initialized: now 1786235798196444 us; error 38 us; skew 500 ppm
I20260809 00:36:38.197789 7164 webserver.cc:460] Webserver started at http://127.21.41.195:37389/ using document root <none> and password file <none>
I20260809 00:36:38.198056 7164 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:38.198127 7164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:38.198287 7164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:38.199402 7164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "44a01f38cd3c457c830909572b8f8b24"
format_stamp: "Formatted at 2026-08-09 00:36:38 on dist-test-slave-zpfg"
server_key: "096b9c3543fb4ca01077feb4520e6867"
server_key_iv: "0af88cba00e8830f7a496271a53a3aab"
server_key_version: "encryptionkey@0"
I20260809 00:36:38.199761 7164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "44a01f38cd3c457c830909572b8f8b24"
format_stamp: "Formatted at 2026-08-09 00:36:38 on dist-test-slave-zpfg"
server_key: "096b9c3543fb4ca01077feb4520e6867"
server_key_iv: "0af88cba00e8830f7a496271a53a3aab"
server_key_version: "encryptionkey@0"
I20260809 00:36:38.201068 7164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:36:38.201817 7178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:38.202042 7164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:38.202164 7164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "44a01f38cd3c457c830909572b8f8b24"
format_stamp: "Formatted at 2026-08-09 00:36:38 on dist-test-slave-zpfg"
server_key: "096b9c3543fb4ca01077feb4520e6867"
server_key_iv: "0af88cba00e8830f7a496271a53a3aab"
server_key_version: "encryptionkey@0"
I20260809 00:36:38.202255 7164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:38.209631 7164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:38.210012 7164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:38.210166 7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:38.210458 7164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:38.210829 7164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:38.210875 7164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:38.210913 7164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:38.210940 7164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:38.216993 7164 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:46593
I20260809 00:36:38.217039 7291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:46593 every 8 connection(s)
I20260809 00:36:38.217363 7164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:36:38.223109 6882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.218084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:59607 (local address 127.21.41.254:41249)
0809 00:36:38.218157 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:38.218159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:38.218430 (+ 271us) server_negotiation.cc:407] Connection header received
0809 00:36:38.219002 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:38.219003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:38.219016 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:38.219075 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:38.219852 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.220383 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.221555 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.221813 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.221900 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.222787 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:38.222788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:38.222788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:38.222794 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:38.222810 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:38.222815 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:38.222882 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:38.222996 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 00:36:38.223028 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:36:38.223335 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.218081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:41249 (local address 127.21.41.195:59607)
0809 00:36:38.218280 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:38.218315 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:38.218882 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:38.219157 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:38.219161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:38.219169 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:38.219669 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.219673 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.220489 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.220491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.221296 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.221298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.221910 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.221911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.222046 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.222617 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:36:38.222618 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:38.222647 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:36:38.222669 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:38.222897 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:38.222903 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:38.222905 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:36:38.222984 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:36:38.223088 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":72,"threads_started":1}
I20260809 00:36:38.223826 7292 heartbeater.cc:347] Connected to a master server at 127.21.41.254:41249
I20260809 00:36:38.223922 7292 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:38.224076 7292 heartbeater.cc:511] Master 127.21.41.254:41249 requested a full tablet report, sending...
I20260809 00:36:38.224395 6840 ts_manager.cc:194] Registered new tserver with Master: 44a01f38cd3c457c830909572b8f8b24 (127.21.41.195:46593)
I20260809 00:36:38.224725 6840 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.195:59607
I20260809 00:36:38.226572 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 7164
I20260809 00:36:38.226652 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:36:38.226892 21671 external_mini_cluster.cc:1428] Setting key 2341b61f69d1668a3a5dd49e7824424d
I20260809 00:36:38.227841 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:36:38.232050 6882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.229269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52108 (local address 127.21.41.254:41249)
0809 00:36:38.229316 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:38.229318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:38.229391 (+ 73us) server_negotiation.cc:407] Connection header received
0809 00:36:38.229442 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:38.229443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:38.229453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:38.229502 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:38.229647 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.230110 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.230952 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.231135 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.231191 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.231844 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:38.231845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:38.231845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:38.231850 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:38.231867 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:38.231871 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:38.231914 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:38.231964 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 00:36:38.231988 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:36:38.233829 6840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:36:38.235404 6840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:36:38.241844 7305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.237997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56414 (local address 127.21.41.194:46447)
0809 00:36:38.238198 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:38.238202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:38.238213 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:36:38.238248 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:38.238250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:38.238264 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:38.238377 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:38.238466 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.239561 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.240784 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.241449 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.241540 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.241636 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:38.241719 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:36:38.241755 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":101,"threads_started":1}
I20260809 00:36:38.241835 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.237795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:46447 (local address 127.0.0.1:56414)
0809 00:36:38.237925 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:38.237933 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:38.237969 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:38.238360 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:38.238361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:38.238363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:38.238408 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.238409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.239667 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.239667 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.240697 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.240700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.241538 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.241539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.241611 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.241624 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 00:36:38.241700 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 00:36:38.241754 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":4,"thread_start_us":65,"threads_started":1}
I20260809 00:36:38.241953 7300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.237530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:46593 (local address 127.0.0.1:35358)
0809 00:36:38.237713 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:38.237752 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:38.237820 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:38.238227 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:38.238228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:38.238231 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:38.238303 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.238304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.239392 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.239393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.240729 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.240730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.241665 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.241666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.241728 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.241733 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:36:38.241764 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 00:36:38.241808 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":36,"thread_start_us":75,"threads_started":1}
I20260809 00:36:38.242097 7304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.237587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:44857 (local address 127.0.0.1:33024)
0809 00:36:38.237905 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:38.237918 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:38.237976 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:38.238224 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:38.238228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:38.238230 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:38.238303 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.238305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.239656 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.239657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.240728 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.240730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.241751 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.241752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.241816 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.241820 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:36:38.241867 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 00:36:38.241899 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":7,"thread_start_us":101,"threads_started":1}
I20260809 00:36:38.242252 7302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.237617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35358 (local address 127.21.41.195:46593)
0809 00:36:38.238087 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:38.238091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:38.238101 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:36:38.238140 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:38.238141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:38.238157 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:38.238230 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:38.238525 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.239297 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.240835 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.241565 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.241877 (+ 312us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.242000 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:38.242101 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 00:36:38.242150 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":87,"threads_started":1}
I20260809 00:36:38.242417 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.237682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33024 (local address 127.21.41.193:44857)
0809 00:36:38.237881 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:38.237886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:38.237991 (+ 105us) server_negotiation.cc:407] Connection header received
0809 00:36:38.238042 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:38.238044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:38.238059 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:38.238134 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:38.238415 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.239570 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.240839 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.241562 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.241634 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.241721 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:38.242261 (+ 540us) server_negotiation.cc:299] Negotiation successful
0809 00:36:38.242308 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":82,"threads_started":1}
I20260809 00:36:38.242985 7226 tablet_service.cc:1467] Processing CreateTablet for tablet e7f82c6072084443aab245b648fbf2a9 (DEFAULT_TABLE table=test-table [id=19bf8f23708e433284e40eb3e82c258c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:38.243080 6964 tablet_service.cc:1467] Processing CreateTablet for tablet e7f82c6072084443aab245b648fbf2a9 (DEFAULT_TABLE table=test-table [id=19bf8f23708e433284e40eb3e82c258c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:38.243141 7095 tablet_service.cc:1467] Processing CreateTablet for tablet e7f82c6072084443aab245b648fbf2a9 (DEFAULT_TABLE table=test-table [id=19bf8f23708e433284e40eb3e82c258c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:38.243364 7226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f82c6072084443aab245b648fbf2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:38.243538 6964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f82c6072084443aab245b648fbf2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:38.243561 7095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f82c6072084443aab245b648fbf2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:38.245891 7306 tablet_bootstrap.cc:492] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: Bootstrap starting.
I20260809 00:36:38.246579 7306 tablet_bootstrap.cc:654] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:38.246810 7306 log.cc:824] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:38.247488 7306 tablet_bootstrap.cc:492] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: No bootstrap required, opened a new log
I20260809 00:36:38.247545 7306 ts_tablet_manager.cc:1397] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:36:38.247504 7309 tablet_bootstrap.cc:492] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: Bootstrap starting.
I20260809 00:36:38.248034 7309 tablet_bootstrap.cc:654] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:38.248322 7309 log.cc:824] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:38.248670 7308 tablet_bootstrap.cc:492] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390: Bootstrap starting.
I20260809 00:36:38.249017 7309 tablet_bootstrap.cc:492] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: No bootstrap required, opened a new log
I20260809 00:36:38.249100 7309 ts_tablet_manager.cc:1397] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:36:38.249572 7306 raft_consensus.cc:348] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a01f38cd3c457c830909572b8f8b24" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 46593 } } peers { permanent_uuid: "0a184e31bbc648229b1c6dd28c3bd390" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44857 } } peers { permanent_uuid: "c8f4b042ca444d7e92ca38dcad571df0" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 46447 } }
I20260809 00:36:38.249722 7306 raft_consensus.cc:374] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:38.249744 7306 raft_consensus.cc:729] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44a01f38cd3c457c830909572b8f8b24, State: Initialized, Role: FOLLOWER
I20260809 00:36:38.249867 7306 consensus_queue.cc:260] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a01f38cd3c457c830909572b8f8b24" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 46593 } } peers { permanent_uuid: "0a184e31bbc648229b1c6dd28c3bd390" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44857 } } peers { permanent_uuid: "c8f4b042ca444d7e92ca38dcad571df0" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 46447 } }
I20260809 00:36:38.250164 7306 ts_tablet_manager.cc:1428] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:36:38.250291 7292 heartbeater.cc:503] Master 127.21.41.254:41249 was elected leader, sending a full tablet report...
I20260809 00:36:38.251519 7309 raft_consensus.cc:348] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a01f38cd3c457c830909572b8f8b24" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 46593 } } peers { permanent_uuid: "0a184e31bbc648229b1c6dd28c3bd390" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44857 } } peers { permanent_uuid: "c8f4b042ca444d7e92ca38dcad571df0" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 46447 } }
I20260809 00:36:38.251668 7309 raft_consensus.cc:374] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:38.251718 7309 raft_consensus.cc:729] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8f4b042ca444d7e92ca38dcad571df0, State: Initialized, Role: FOLLOWER
I20260809 00:36:38.251833 7309 consensus_queue.cc:260] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a01f38cd3c457c830909572b8f8b24" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 46593 } } peers { permanent_uuid: "0a184e31bbc648229b1c6dd28c3bd390" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44857 } } peers { permanent_uuid: "c8f4b042ca444d7e92ca38dcad571df0" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 46447 } }
I20260809 00:36:38.252064 7309 ts_tablet_manager.cc:1428] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:36:38.252121 7161 heartbeater.cc:503] Master 127.21.41.254:41249 was elected leader, sending a full tablet report...
I20260809 00:36:38.252750 7308 tablet_bootstrap.cc:654] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:38.253093 7308 log.cc:824] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:38.253878 7308 tablet_bootstrap.cc:492] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390: No bootstrap required, opened a new log
I20260809 00:36:38.253963 7308 ts_tablet_manager.cc:1397] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 00:36:38.256017 7308 raft_consensus.cc:348] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a01f38cd3c457c830909572b8f8b24" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 46593 } } peers { permanent_uuid: "0a184e31bbc648229b1c6dd28c3bd390" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44857 } } peers { permanent_uuid: "c8f4b042ca444d7e92ca38dcad571df0" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 46447 } }
I20260809 00:36:38.256184 7308 raft_consensus.cc:374] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:38.256232 7308 raft_consensus.cc:729] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a184e31bbc648229b1c6dd28c3bd390, State: Initialized, Role: FOLLOWER
I20260809 00:36:38.256354 7308 consensus_queue.cc:260] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a01f38cd3c457c830909572b8f8b24" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 46593 } } peers { permanent_uuid: "0a184e31bbc648229b1c6dd28c3bd390" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44857 } } peers { permanent_uuid: "c8f4b042ca444d7e92ca38dcad571df0" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 46447 } }
I20260809 00:36:38.256609 7308 ts_tablet_manager.cc:1428] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:36:38.256683 7030 heartbeater.cc:503] Master 127.21.41.254:41249 was elected leader, sending a full tablet report...
W20260809 00:36:38.344030 7162 tablet.cc:2369] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:36:38.458679 7312 raft_consensus.cc:482] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:36:38.458833 7312 raft_consensus.cc:504] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a01f38cd3c457c830909572b8f8b24" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 46593 } } peers { permanent_uuid: "0a184e31bbc648229b1c6dd28c3bd390" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44857 } } peers { permanent_uuid: "c8f4b042ca444d7e92ca38dcad571df0" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 46447 } }
I20260809 00:36:38.459214 7312 leader_election.cc:288] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44a01f38cd3c457c830909572b8f8b24 (127.21.41.195:46593), 0a184e31bbc648229b1c6dd28c3bd390 (127.21.41.193:44857)
W20260809 00:36:38.461032 7031 tablet.cc:2369] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:36:38.462244 7302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.459528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:36031 (local address 127.21.41.195:46593)
0809 00:36:38.459621 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:38.459623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:38.459633 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:36:38.459671 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:38.459672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:38.459679 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:38.459750 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:38.459924 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.460416 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.461529 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.461902 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.461990 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.461998 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:38.462133 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 00:36:38.462169 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 00:36:38.462283 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.459430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:46593 (local address 127.21.41.194:36031)
0809 00:36:38.459492 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:38.459505 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:38.459562 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:38.459756 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:38.459757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:38.459759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:38.459849 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.459850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.460506 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.460507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.461437 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.461440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.461988 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.461989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.462058 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.462061 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:36:38.462128 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 00:36:38.462169 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":28}
I20260809 00:36:38.462294 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.459527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:38327 (local address 127.21.41.193:44857)
0809 00:36:38.459580 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:38.459582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:38.459728 (+ 146us) server_negotiation.cc:407] Connection header received
0809 00:36:38.459803 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:38.459804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:38.459817 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:38.459894 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:38.460056 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.460516 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.461557 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.461913 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.462000 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.462008 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:38.462186 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:36:38.462225 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:36:38.462438 7315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.459429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:44857 (local address 127.21.41.194:38327)
0809 00:36:38.459635 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:38.459646 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:38.459746 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:38.459907 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:38.459907 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:38.459908 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:38.459974 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.459975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.460607 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.460608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.461458 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:38.461460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.462002 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.462003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:38.462094 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.462097 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:36:38.462176 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:36:38.462214 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":22,"thread_start_us":101,"threads_started":1}
I20260809 00:36:38.462673 7246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f82c6072084443aab245b648fbf2a9" candidate_uuid: "c8f4b042ca444d7e92ca38dcad571df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44a01f38cd3c457c830909572b8f8b24" is_pre_election: true
I20260809 00:36:38.462769 6984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f82c6072084443aab245b648fbf2a9" candidate_uuid: "c8f4b042ca444d7e92ca38dcad571df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a184e31bbc648229b1c6dd28c3bd390" is_pre_election: true
I20260809 00:36:38.462845 7246 raft_consensus.cc:2454] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8f4b042ca444d7e92ca38dcad571df0 in term 0.
I20260809 00:36:38.462893 6984 raft_consensus.cc:2454] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8f4b042ca444d7e92ca38dcad571df0 in term 0.
I20260809 00:36:38.463096 7049 leader_election.cc:302] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a184e31bbc648229b1c6dd28c3bd390, c8f4b042ca444d7e92ca38dcad571df0; no voters:
I20260809 00:36:38.463240 7312 raft_consensus.cc:2781] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:36:38.463339 7312 raft_consensus.cc:482] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:36:38.463380 7312 raft_consensus.cc:3037] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:38.464102 7312 raft_consensus.cc:504] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a01f38cd3c457c830909572b8f8b24" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 46593 } } peers { permanent_uuid: "0a184e31bbc648229b1c6dd28c3bd390" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44857 } } peers { permanent_uuid: "c8f4b042ca444d7e92ca38dcad571df0" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 46447 } }
I20260809 00:36:38.464280 7312 leader_election.cc:288] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [CANDIDATE]: Term 1 election: Requested vote from peers 44a01f38cd3c457c830909572b8f8b24 (127.21.41.195:46593), 0a184e31bbc648229b1c6dd28c3bd390 (127.21.41.193:44857)
I20260809 00:36:38.464466 7246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f82c6072084443aab245b648fbf2a9" candidate_uuid: "c8f4b042ca444d7e92ca38dcad571df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44a01f38cd3c457c830909572b8f8b24"
I20260809 00:36:38.464496 6984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f82c6072084443aab245b648fbf2a9" candidate_uuid: "c8f4b042ca444d7e92ca38dcad571df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a184e31bbc648229b1c6dd28c3bd390"
I20260809 00:36:38.464566 6984 raft_consensus.cc:3037] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:38.464571 7246 raft_consensus.cc:3037] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:38.465286 7246 raft_consensus.cc:2454] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8f4b042ca444d7e92ca38dcad571df0 in term 1.
I20260809 00:36:38.465294 6984 raft_consensus.cc:2454] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8f4b042ca444d7e92ca38dcad571df0 in term 1.
I20260809 00:36:38.465497 7049 leader_election.cc:302] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a184e31bbc648229b1c6dd28c3bd390, c8f4b042ca444d7e92ca38dcad571df0; no voters:
I20260809 00:36:38.465583 7312 raft_consensus.cc:2781] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:38.465751 7312 raft_consensus.cc:686] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 1 LEADER]: Becoming Leader. State: Replica: c8f4b042ca444d7e92ca38dcad571df0, State: Running, Role: LEADER
I20260809 00:36:38.465862 7312 consensus_queue.cc:237] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a01f38cd3c457c830909572b8f8b24" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 46593 } } peers { permanent_uuid: "0a184e31bbc648229b1c6dd28c3bd390" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44857 } } peers { permanent_uuid: "c8f4b042ca444d7e92ca38dcad571df0" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 46447 } }
I20260809 00:36:38.466686 6840 catalog_manager.cc:5495] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8f4b042ca444d7e92ca38dcad571df0 (127.21.41.194). New cstate: current_term: 1 leader_uuid: "c8f4b042ca444d7e92ca38dcad571df0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a01f38cd3c457c830909572b8f8b24" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 46593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a184e31bbc648229b1c6dd28c3bd390" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8f4b042ca444d7e92ca38dcad571df0" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 46447 } health_report { overall_health: HEALTHY } } }
W20260809 00:36:38.467878 7293 tablet.cc:2369] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:36:38.471741 7305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.469291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56416 (local address 127.21.41.194:46447)
0809 00:36:38.469334 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:38.469336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:38.469345 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:36:38.469397 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:38.469398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:38.469414 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:38.469497 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:36:38.469626 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.470083 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.470958 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:38.471151 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:38.471237 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:38.471419 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:36:38.471517 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:36:38.471615 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:38.471633 (+ 18us) server_negotiation.cc:299] Negotiation successful
0809 00:36:38.471653 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:36:38.473659 7246 raft_consensus.cc:1261] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24 [term 1 FOLLOWER]: Refusing update from remote peer c8f4b042ca444d7e92ca38dcad571df0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:38.473770 6984 raft_consensus.cc:1261] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390 [term 1 FOLLOWER]: Refusing update from remote peer c8f4b042ca444d7e92ca38dcad571df0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:38.474102 7316 consensus_queue.cc:1035] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a184e31bbc648229b1c6dd28c3bd390" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 44857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:38.474290 7312 consensus_queue.cc:1035] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44a01f38cd3c457c830909572b8f8b24" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 46593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:38.476599 7319 mvcc.cc:205] Tried to move back new op lower bound from 7316421830544846848 to 7316421830516699136. Current Snapshot: MvccSnapshot[applied={T|T < 7316421830544846848}]
I20260809 00:36:38.476781 7321 mvcc.cc:205] Tried to move back new op lower bound from 7316421830544846848 to 7316421830516699136. Current Snapshot: MvccSnapshot[applied={T|T < 7316421830544846848}]
I20260809 00:36:38.477562 7320 mvcc.cc:205] Tried to move back new op lower bound from 7316421830544846848 to 7316421830516699136. Current Snapshot: MvccSnapshot[applied={T|T < 7316421830544846848}]
I20260809 00:36:38.479910 6841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:38.480017 6841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:38.480859 6841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f098b240b6649b2a85b5ec8ae7e5c53: Sending DeleteTablet for 3 replicas of tablet e7f82c6072084443aab245b648fbf2a9
I20260809 00:36:38.481222 6964 tablet_service.cc:1514] Processing DeleteTablet for tablet e7f82c6072084443aab245b648fbf2a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:38 UTC) from {username='slave'} at 127.0.0.1:33024
I20260809 00:36:38.481230 7226 tablet_service.cc:1514] Processing DeleteTablet for tablet e7f82c6072084443aab245b648fbf2a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:38 UTC) from {username='slave'} at 127.0.0.1:35358
I20260809 00:36:38.481508 7095 tablet_service.cc:1514] Processing DeleteTablet for tablet e7f82c6072084443aab245b648fbf2a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:38 UTC) from {username='slave'} at 127.0.0.1:56414
I20260809 00:36:38.481617 7329 tablet_replica.cc:330] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390: stopping tablet replica
I20260809 00:36:38.481716 7329 raft_consensus.cc:2229] T e7f82c6072084443aab245b648fbf2a9 P 0a184e31bbc648229b1c6dd28c3bd390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:38.481726 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 6902
I20260809 00:36:38.482374 7330 tablet_replica.cc:330] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: stopping tablet replica
I20260809 00:36:38.482488 7330 raft_consensus.cc:2229] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:36:38.482625 7330 raft_consensus.cc:2258] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:38.483060 7328 tablet_replica.cc:330] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: stopping tablet replica
I20260809 00:36:38.483076 7330 ts_tablet_manager.cc:1905] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:36:38.483235 7328 raft_consensus.cc:2229] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:38.483389 7328 raft_consensus.cc:2258] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:38.483872 7328 ts_tablet_manager.cc:1905] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:36:38.484881 7330 ts_tablet_manager.cc:1918] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:38.485006 7330 log.cc:1196] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/e7f82c6072084443aab245b648fbf2a9
I20260809 00:36:38.485379 7330 ts_tablet_manager.cc:1939] T e7f82c6072084443aab245b648fbf2a9 P c8f4b042ca444d7e92ca38dcad571df0: Deleting consensus metadata
I20260809 00:36:38.485579 7328 ts_tablet_manager.cc:1918] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:38.485682 7328 log.cc:1196] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/e7f82c6072084443aab245b648fbf2a9
I20260809 00:36:38.485926 6826 catalog_manager.cc:4841] TS c8f4b042ca444d7e92ca38dcad571df0 (127.21.41.194:46447): tablet e7f82c6072084443aab245b648fbf2a9 (table test-table [id=19bf8f23708e433284e40eb3e82c258c]) successfully deleted
I20260809 00:36:38.485980 7328 ts_tablet_manager.cc:1939] T e7f82c6072084443aab245b648fbf2a9 P 44a01f38cd3c457c830909572b8f8b24: Deleting consensus metadata
I20260809 00:36:38.486419 6827 catalog_manager.cc:4841] TS 44a01f38cd3c457c830909572b8f8b24 (127.21.41.195:46593): tablet e7f82c6072084443aab245b648fbf2a9 (table test-table [id=19bf8f23708e433284e40eb3e82c258c]) successfully deleted
W20260809 00:36:38.488181 6826 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:44857 (error 108)
I20260809 00:36:38.488478 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 7033
I20260809 00:36:38.488662 7304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:38.488518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:44857 (local address 127.0.0.1:33028)
0809 00:36:38.488580 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:38.488621 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:44857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 00:36:38.490167 6826 catalog_manager.cc:4584] TS 0a184e31bbc648229b1c6dd28c3bd390 (127.21.41.193:44857): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e7f82c6072084443aab245b648fbf2a9: Network error: Client connection negotiation failed: client connection to 127.21.41.193:44857: connect: Connection refused (error 111)
I20260809 00:36:38.493929 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 7164
I20260809 00:36:38.498505 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 6810
2026-08-09T00:36:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (839 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:38 dist-test-slave-zpfg krb5kdc[7334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:38 dist-test-slave-zpfg krb5kdc[7334](info): set up 2 sockets
Aug 09 00:36:38 dist-test-slave-zpfg krb5kdc[7334](info): commencing operation
krb5kdc: starting...
W20260809 00:36:39.548281 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:39 dist-test-slave-zpfg krb5kdc[7334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:36:39.673672 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:44169
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:44169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:39.760514 7347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:39.760736 7347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:39.760761 7347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:39.761868 7347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:39.761904 7347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:39.761919 7347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:39.761931 7347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:39.761943 7347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:39.763381 7347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:44169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:44169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:39.763586 7347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:39.763798 7347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:39.768719 7353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:39.768714 7355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:39.768683 7352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:39.768847 7347 server_base.cc:1034] running on GCE node
I20260809 00:36:39.769269 7347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:39.769564 7347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:39.770731 7347 hybrid_clock.cc:648] HybridClock initialized: now 1786235799770724 us; error 40 us; skew 500 ppm
Aug 09 00:36:39 dist-test-slave-zpfg krb5kdc[7334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:39.773130 7347 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:36:39.773672 7347 webserver.cc:460] Webserver started at http://127.21.41.254:44291/ using document root <none> and password file <none>
I20260809 00:36:39.773921 7347 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:39.773968 7347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:39.774115 7347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:39.775321 7347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "8370bed9e33d4118b0707ae9da15666a"
format_stamp: "Formatted at 2026-08-09 00:36:39 on dist-test-slave-zpfg"
server_key: "653394647c0274f5e7b5ea4cdb1523f9"
server_key_iv: "24f343af60ece3da9673c8ec0a308c0c"
server_key_version: "encryptionkey@0"
I20260809 00:36:39.775708 7347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "8370bed9e33d4118b0707ae9da15666a"
format_stamp: "Formatted at 2026-08-09 00:36:39 on dist-test-slave-zpfg"
server_key: "653394647c0274f5e7b5ea4cdb1523f9"
server_key_iv: "24f343af60ece3da9673c8ec0a308c0c"
server_key_version: "encryptionkey@0"
I20260809 00:36:39.777089 7347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:36:39.777949 7362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:39.778132 7347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:36:39.778247 7347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "8370bed9e33d4118b0707ae9da15666a"
format_stamp: "Formatted at 2026-08-09 00:36:39 on dist-test-slave-zpfg"
server_key: "653394647c0274f5e7b5ea4cdb1523f9"
server_key_iv: "24f343af60ece3da9673c8ec0a308c0c"
server_key_version: "encryptionkey@0"
I20260809 00:36:39.778327 7347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:39.787387 7347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:39.789446 7347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:39.789662 7347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:39.793994 7347 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:44169
I20260809 00:36:39.794035 7414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:44169 every 8 connection(s)
I20260809 00:36:39.794503 7347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:39.795614 7415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:39.798435 7415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a: Bootstrap starting.
I20260809 00:36:39.799021 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 7347
I20260809 00:36:39.799109 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:39.799160 7415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:39.799331 21671 external_mini_cluster.cc:1428] Setting key 4f19be4e56285edfcd9fc066f13f09d3
I20260809 00:36:39.799574 7415 log.cc:824] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:39.800680 7415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a: No bootstrap required, opened a new log
I20260809 00:36:39.802738 7415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370bed9e33d4118b0707ae9da15666a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 44169 } }
I20260809 00:36:39.802909 7415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:39.802981 7415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8370bed9e33d4118b0707ae9da15666a, State: Initialized, Role: FOLLOWER
I20260809 00:36:39.803165 7415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370bed9e33d4118b0707ae9da15666a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 44169 } }
I20260809 00:36:39.803267 7415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:39.803313 7415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:39.803372 7415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:39.803957 7415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370bed9e33d4118b0707ae9da15666a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 44169 } }
I20260809 00:36:39.804107 7415 leader_election.cc:302] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8370bed9e33d4118b0707ae9da15666a; no voters:
I20260809 00:36:39.804296 7415 leader_election.cc:288] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:39.804419 7420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:39.804603 7420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [term 1 LEADER]: Becoming Leader. State: Replica: 8370bed9e33d4118b0707ae9da15666a, State: Running, Role: LEADER
I20260809 00:36:39.804816 7420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370bed9e33d4118b0707ae9da15666a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 44169 } }
I20260809 00:36:39.805320 7421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8370bed9e33d4118b0707ae9da15666a. Latest consensus state: current_term: 1 leader_uuid: "8370bed9e33d4118b0707ae9da15666a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370bed9e33d4118b0707ae9da15666a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 44169 } } }
I20260809 00:36:39.805413 7421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:39.805346 7422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8370bed9e33d4118b0707ae9da15666a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370bed9e33d4118b0707ae9da15666a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 44169 } } }
I20260809 00:36:39.805606 7422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:39.805634 7424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:39.805797 7415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:39.806211 7424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 00:36:39 dist-test-slave-zpfg krb5kdc[7334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:39.808123 7424 catalog_manager.cc:1347] Generated new cluster ID: de2b0224d1474892854ceb3d8d2e1917
I20260809 00:36:39.808202 7424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:39.811844 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:39.799929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.21.41.254:44169)
0809 00:36:39.800161 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:39.800171 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:39.800206 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:36:39.800738 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:39.800745 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:39.800849 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:39.801041 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:39.801730 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:39.802754 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:39.803769 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:39.804351 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:39.804477 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:39.807602 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:39.807643 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:39.807643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:39.807658 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:39.809679 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:39.810047 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:39.810050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:39.810050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:39.810159 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:39.810373 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:39.810374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:39.810374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:39.810661 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:39.810814 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:39.811318 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 00:36:39.811452 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
I20260809 00:36:39.812765 7424 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:39.813438 7424 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:39.820402 7424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a: Generated new TSK 0
I20260809 00:36:39.820580 7424 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:36:39.853155 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:44169
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:39.962456 7441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:39.962713 7441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:39.962761 7441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:39.964072 7441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:39.964152 7441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:39.964203 7441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:36:39.965750 7441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:44169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:39.966023 7441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:39.966264 7441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:39.971578 7446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:39.971614 7449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:39.971614 7447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:39.971973 7441 server_base.cc:1034] running on GCE node
I20260809 00:36:39.972193 7441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:39.972457 7441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:39.973630 7441 hybrid_clock.cc:648] HybridClock initialized: now 1786235799973618 us; error 34 us; skew 500 ppm
Aug 09 00:36:39 dist-test-slave-zpfg krb5kdc[7334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:39.976281 7441 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:36:39.976787 7441 webserver.cc:460] Webserver started at http://127.21.41.193:40645/ using document root <none> and password file <none>
I20260809 00:36:39.977034 7441 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:39.977103 7441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:39.977237 7441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:39.978317 7441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "af9ef30f632c4cb4ac480c91de266a6b"
format_stamp: "Formatted at 2026-08-09 00:36:39 on dist-test-slave-zpfg"
server_key: "0cfd01adaf3c5ed3a2e18bc167069928"
server_key_iv: "501a7097dc782dcf2a4fffbf04db8179"
server_key_version: "encryptionkey@0"
I20260809 00:36:39.978783 7441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "af9ef30f632c4cb4ac480c91de266a6b"
format_stamp: "Formatted at 2026-08-09 00:36:39 on dist-test-slave-zpfg"
server_key: "0cfd01adaf3c5ed3a2e18bc167069928"
server_key_iv: "501a7097dc782dcf2a4fffbf04db8179"
server_key_version: "encryptionkey@0"
I20260809 00:36:39.980191 7441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:39.981154 7456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:39.981334 7441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:39.981441 7441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "af9ef30f632c4cb4ac480c91de266a6b"
format_stamp: "Formatted at 2026-08-09 00:36:39 on dist-test-slave-zpfg"
server_key: "0cfd01adaf3c5ed3a2e18bc167069928"
server_key_iv: "501a7097dc782dcf2a4fffbf04db8179"
server_key_version: "encryptionkey@0"
I20260809 00:36:39.981527 7441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:39.996381 7441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:39.999032 7441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:39.999220 7441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:39.999497 7441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:39.999846 7441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:39.999908 7441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:39.999951 7441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:39.999981 7441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:40.006841 7441 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:35747
I20260809 00:36:40.006887 7570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:35747 every 8 connection(s)
I20260809 00:36:40.007323 7441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:36:40.009541 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 7441
I20260809 00:36:40.009642 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:36:40.009876 21671 external_mini_cluster.cc:1428] Setting key 26d72b87851674f988cba1eb4d2cb302
Aug 09 00:36:40 dist-test-slave-zpfg krb5kdc[7334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:40.018914 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.008100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:41303 (local address 127.21.41.254:44169)
0809 00:36:40.008194 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.008196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.008414 (+ 218us) server_negotiation.cc:407] Connection header received
0809 00:36:40.009162 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.009163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.009178 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.009244 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:40.010785 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.011330 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.012874 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.013088 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.013143 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.015820 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:40.015822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:40.015823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:40.015831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:40.017383 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:40.017825 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:40.017826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:40.017827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:40.017891 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:40.018156 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:40.018157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:40.018157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:40.018288 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:40.018420 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.018704 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.018814 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 00:36:40.019119 7573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.008056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:44169 (local address 127.21.41.193:41303)
0809 00:36:40.008246 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:40.008307 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:40.009005 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:40.009326 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:40.009330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:40.010232 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:40.010553 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.010559 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.011446 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.011448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.012751 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.012754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.013183 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.013185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.013359 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.013900 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:40.013911 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:40.015700 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:40.017478 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:40.017487 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:40.017489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:40.017726 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:40.017962 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:40.017962 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:40.017963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:40.018081 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:40.018404 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:40.018412 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:40.018562 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 00:36:40.018684 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 00:36:40.018867 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260809 00:36:40.019622 7571 heartbeater.cc:347] Connected to a master server at 127.21.41.254:44169
I20260809 00:36:40.019724 7571 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:40.019923 7571 heartbeater.cc:511] Master 127.21.41.254:44169 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
I20260809 00:36:40.020535 7379 ts_manager.cc:194] Registered new tserver with Master: af9ef30f632c4cb4ac480c91de266a6b (127.21.41.193:35747)
I20260809 00:36:40.021294 7379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:41303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:36:40.041376 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:44169
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:40.126459 7576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:40.126668 7576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:40.126694 7576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:40.127810 7576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:40.127843 7576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:40.127869 7576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:36:40.129237 7576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:44169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:40.129415 7576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:40.129588 7576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:40.134837 7581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:40.134974 7584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:40.134837 7582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:40.135040 7576 server_base.cc:1034] running on GCE node
I20260809 00:36:40.135270 7576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:40.135546 7576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:40.136758 7576 hybrid_clock.cc:648] HybridClock initialized: now 1786235800136745 us; error 41 us; skew 500 ppm
Aug 09 00:36:40 dist-test-slave-zpfg krb5kdc[7334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235800, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:40.139499 7576 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:36:40.140044 7576 webserver.cc:460] Webserver started at http://127.21.41.194:44039/ using document root <none> and password file <none>
I20260809 00:36:40.140278 7576 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:40.140347 7576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:40.140506 7576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:40.141558 7576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "807f10cef6a44d399b777185608b3f36"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "2fda64987f7b18932b550365749ca84d"
server_key_iv: "ebdb0c045dd739114ee3f05e77d6e615"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.141938 7576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "807f10cef6a44d399b777185608b3f36"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "2fda64987f7b18932b550365749ca84d"
server_key_iv: "ebdb0c045dd739114ee3f05e77d6e615"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.143356 7576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:36:40.144112 7591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:40.144305 7576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:36:40.144413 7576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "807f10cef6a44d399b777185608b3f36"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "2fda64987f7b18932b550365749ca84d"
server_key_iv: "ebdb0c045dd739114ee3f05e77d6e615"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.144487 7576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:40.156286 7576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:40.158648 7576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:40.158815 7576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:40.159058 7576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:40.159382 7576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:40.159416 7576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:40.159438 7576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:40.159487 7576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:40.165139 7576 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:35409
I20260809 00:36:40.165212 7704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:35409 every 8 connection(s)
I20260809 00:36:40.165539 7576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:36:40.166935 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 7576
I20260809 00:36:40.167023 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:36:40.167212 21671 external_mini_cluster.cc:1428] Setting key 05f04eb2555132b9017f294f5eb68267
Aug 09 00:36:40 dist-test-slave-zpfg krb5kdc[7334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235800, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:40.175876 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.166308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:50157 (local address 127.21.41.254:44169)
0809 00:36:40.166404 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.166407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.166592 (+ 185us) server_negotiation.cc:407] Connection header received
0809 00:36:40.167108 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.167110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.167122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.167178 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:40.168388 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.168882 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.169923 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.170121 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.170211 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.172732 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:40.172734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:40.172734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:40.172742 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:40.174444 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:40.174866 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:40.174867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:40.174868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:40.174932 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:40.175174 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:40.175175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:40.175175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:40.175327 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:40.175416 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.175673 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.175756 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 00:36:40.175963 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.166253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:44169 (local address 127.21.41.194:50157)
0809 00:36:40.166465 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:40.166509 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:40.167013 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:40.167249 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:40.167253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:40.168068 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:40.168286 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.168291 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.168982 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.168983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.169808 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.169810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.170222 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.170223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.170409 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.170962 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:40.170972 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:40.172634 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:40.174534 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:40.174541 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:40.174543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:40.174757 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:40.175008 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:40.175009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:40.175009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:40.175105 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:40.175408 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:40.175412 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:40.175525 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 00:36:40.175607 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 00:36:40.175738 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":116,"threads_started":1}
I20260809 00:36:40.176468 7705 heartbeater.cc:347] Connected to a master server at 127.21.41.254:44169
I20260809 00:36:40.176584 7705 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:40.176787 7705 heartbeater.cc:511] Master 127.21.41.254:44169 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
I20260809 00:36:40.177217 7379 ts_manager.cc:194] Registered new tserver with Master: 807f10cef6a44d399b777185608b3f36 (127.21.41.194:35409)
I20260809 00:36:40.177578 7379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:50157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:36:40.198639 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:44169
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:45589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:40.283561 7710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:40.283840 7710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:40.283886 7710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:40.285188 7710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:40.285257 7710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:40.285305 7710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:36:40.286948 7710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:45589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:44169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:40.287231 7710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:40.287480 7710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:40.292806 7716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:40.292764 7718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:40.292760 7715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:40.293123 7710 server_base.cc:1034] running on GCE node
I20260809 00:36:40.293287 7710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:40.293510 7710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:40.294648 7710 hybrid_clock.cc:648] HybridClock initialized: now 1786235800294637 us; error 27 us; skew 500 ppm
Aug 09 00:36:40 dist-test-slave-zpfg krb5kdc[7334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235800, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:40.297197 7710 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:36:40.297730 7710 webserver.cc:460] Webserver started at http://127.21.41.195:34745/ using document root <none> and password file <none>
I20260809 00:36:40.297976 7710 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:40.298022 7710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:40.298185 7710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:40.299363 7710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "7bee1f94fc354faeabebd6bd1d917829"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "ce49e9ce747897746cdec6e37cdb75dd"
server_key_iv: "f165284fa8a22749bd9bd2231247d194"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.299741 7710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "7bee1f94fc354faeabebd6bd1d917829"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "ce49e9ce747897746cdec6e37cdb75dd"
server_key_iv: "f165284fa8a22749bd9bd2231247d194"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.301117 7710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:36:40.301855 7725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:40.302028 7710 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:36:40.302136 7710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "7bee1f94fc354faeabebd6bd1d917829"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "ce49e9ce747897746cdec6e37cdb75dd"
server_key_iv: "f165284fa8a22749bd9bd2231247d194"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.302214 7710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:40.333608 7710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:40.335867 7710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:40.336046 7710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:40.336295 7710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:40.336628 7710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:40.336687 7710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:40.336732 7710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:40.336758 7710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:40.343135 7710 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:38991
I20260809 00:36:40.343180 7838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:38991 every 8 connection(s)
I20260809 00:36:40.343509 7710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:36:40.344233 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 7710
I20260809 00:36:40.344326 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:36:40.344554 21671 external_mini_cluster.cc:1428] Setting key e463c3e45e52bd5e46f4ecc956f15ff7
Aug 09 00:36:40 dist-test-slave-zpfg krb5kdc[7334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235800, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:40.353190 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.344287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:41643 (local address 127.21.41.254:44169)
0809 00:36:40.344341 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.344344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.344626 (+ 282us) server_negotiation.cc:407] Connection header received
0809 00:36:40.345286 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.345287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.345301 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.345362 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:40.346455 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.346961 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.348047 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.348246 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.348356 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.350554 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:40.350555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:40.350555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:40.350564 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:40.351779 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:40.352189 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:40.352190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:40.352190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:40.352251 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:40.352516 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:40.352517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:40.352517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:40.352633 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:40.352713 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.353011 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.353092 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:36:40.353382 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.344293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:44169 (local address 127.21.41.195:41643)
0809 00:36:40.344490 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:40.344530 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:40.345199 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:40.345428 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:40.345432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:40.346106 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:40.346302 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.346307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.347062 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.347063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.347936 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.347938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.348359 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.348360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.348509 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.348997 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:40.349005 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:40.350461 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:40.351875 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:40.351882 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:40.351884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:40.352108 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:40.352331 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:40.352332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:40.352332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:40.352446 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:40.352704 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:40.352711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:40.352871 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 00:36:40.352988 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 00:36:40.353149 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":86,"threads_started":1}
I20260809 00:36:40.353904 7839 heartbeater.cc:347] Connected to a master server at 127.21.41.254:44169
I20260809 00:36:40.353987 7839 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:40.354185 7839 heartbeater.cc:511] Master 127.21.41.254:44169 requested a full tablet report, sending...
I20260809 00:36:40.354694 7379 ts_manager.cc:194] Registered new tserver with Master: 7bee1f94fc354faeabebd6bd1d917829 (127.21.41.195:38991)
I20260809 00:36:40.355082 7379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:41643
I20260809 00:36:40.356493 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:36:40.364288 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.357968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.21.41.254:44169)
0809 00:36:40.358028 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.358030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.358176 (+ 146us) server_negotiation.cc:407] Connection header received
0809 00:36:40.358230 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.358231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.358239 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.358284 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:40.358995 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.359468 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.360214 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.360402 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.360452 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.361727 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:40.361728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:40.361728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:40.361734 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:40.363087 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:40.363449 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:40.363450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:40.363451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:40.363500 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:40.363782 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:40.363783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:40.363783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:40.363901 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:40.363981 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.364112 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.364164 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:36:40.366196 7379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:36:40.367859 7379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:36:40.374977 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.370533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.21.41.193:35747)
0809 00:36:40.370951 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.370957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.370967 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:36:40.371010 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.371012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.371028 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.371107 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:40.371327 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.372565 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.373876 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.374464 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.374570 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.374639 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.374833 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.374884 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":84,"threads_started":1}
I20260809 00:36:40.374991 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.370442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:35747 (local address 127.0.0.1:50312)
0809 00:36:40.370639 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:40.370684 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:40.370747 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:40.371151 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:40.371155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:40.371158 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:40.371235 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.371237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.372677 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.372678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.373787 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.373789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.374573 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.374574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.374660 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.374677 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 00:36:40.374810 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 00:36:40.374867 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":28,"thread_start_us":60,"threads_started":1}
I20260809 00:36:40.374993 7854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.370643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:35409 (local address 127.0.0.1:39376)
0809 00:36:40.370855 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:40.370864 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:40.370900 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:40.371234 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:40.371234 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:40.371236 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:40.371289 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.371291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.372705 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.372707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.373664 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.373666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.374655 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.374656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.374746 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.374750 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:36:40.374783 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 00:36:40.374865 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":6,"thread_start_us":67,"threads_started":1}
I20260809 00:36:40.375070 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.370572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37760 (local address 127.21.41.195:38991)
0809 00:36:40.371275 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.371280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.371289 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:36:40.371324 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.371326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.371341 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.371439 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:40.371652 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.372325 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.373816 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.374549 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.374624 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.374680 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.374936 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.374979 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":92,"threads_started":1}
I20260809 00:36:40.375224 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.370480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:38991 (local address 127.0.0.1:37760)
0809 00:36:40.370709 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:40.370720 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:40.370786 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:40.371514 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:40.371515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:40.371517 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:40.371570 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.371572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.372444 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.372445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.373739 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.373742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.374710 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.374711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.374786 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.374790 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:36:40.374836 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 00:36:40.374880 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":3,"thread_start_us":83,"threads_started":1}
I20260809 00:36:40.375829 7855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.370874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39376 (local address 127.21.41.194:35409)
0809 00:36:40.371059 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.371064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.371073 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:36:40.371113 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.371116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.371133 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.371432 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:40.371506 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.372564 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.373763 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.374560 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.375487 (+ 927us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.375612 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.375656 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.375708 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260809 00:36:40.376089 7773 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0336c6949b4293832c1eb35f8192e4 (DEFAULT_TABLE table=test-table [id=0f870bb3e700478b9e269a1b304d4b58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:40.376088 7504 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0336c6949b4293832c1eb35f8192e4 (DEFAULT_TABLE table=test-table [id=0f870bb3e700478b9e269a1b304d4b58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:40.376467 7773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0336c6949b4293832c1eb35f8192e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:40.376466 7504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0336c6949b4293832c1eb35f8192e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:40.376590 7639 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0336c6949b4293832c1eb35f8192e4 (DEFAULT_TABLE table=test-table [id=0f870bb3e700478b9e269a1b304d4b58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:40.376932 7639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0336c6949b4293832c1eb35f8192e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:40.379534 7856 tablet_bootstrap.cc:492] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: Bootstrap starting.
I20260809 00:36:40.380174 7857 tablet_bootstrap.cc:492] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b: Bootstrap starting.
I20260809 00:36:40.380291 7856 tablet_bootstrap.cc:654] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:40.380822 7857 tablet_bootstrap.cc:654] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:40.380776 7856 log.cc:824] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:40.381428 7857 log.cc:824] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:40.381705 7856 tablet_bootstrap.cc:492] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: No bootstrap required, opened a new log
I20260809 00:36:40.381785 7856 ts_tablet_manager.cc:1397] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:36:40.381770 7858 tablet_bootstrap.cc:492] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829: Bootstrap starting.
I20260809 00:36:40.382321 7857 tablet_bootstrap.cc:492] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b: No bootstrap required, opened a new log
I20260809 00:36:40.382463 7858 tablet_bootstrap.cc:654] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:40.382485 7857 ts_tablet_manager.cc:1397] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.001s
I20260809 00:36:40.383028 7858 log.cc:824] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:40.383689 7856 raft_consensus.cc:348] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9ef30f632c4cb4ac480c91de266a6b" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35747 } } peers { permanent_uuid: "807f10cef6a44d399b777185608b3f36" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 35409 } } peers { permanent_uuid: "7bee1f94fc354faeabebd6bd1d917829" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38991 } }
I20260809 00:36:40.383903 7856 raft_consensus.cc:374] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:40.383955 7856 raft_consensus.cc:729] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 807f10cef6a44d399b777185608b3f36, State: Initialized, Role: FOLLOWER
I20260809 00:36:40.384071 7856 consensus_queue.cc:260] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9ef30f632c4cb4ac480c91de266a6b" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35747 } } peers { permanent_uuid: "807f10cef6a44d399b777185608b3f36" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 35409 } } peers { permanent_uuid: "7bee1f94fc354faeabebd6bd1d917829" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38991 } }
I20260809 00:36:40.384409 7856 ts_tablet_manager.cc:1428] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:36:40.384428 7705 heartbeater.cc:503] Master 127.21.41.254:44169 was elected leader, sending a full tablet report...
I20260809 00:36:40.384543 7857 raft_consensus.cc:348] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9ef30f632c4cb4ac480c91de266a6b" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35747 } } peers { permanent_uuid: "807f10cef6a44d399b777185608b3f36" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 35409 } } peers { permanent_uuid: "7bee1f94fc354faeabebd6bd1d917829" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38991 } }
I20260809 00:36:40.384719 7857 raft_consensus.cc:374] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:40.384757 7857 raft_consensus.cc:729] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af9ef30f632c4cb4ac480c91de266a6b, State: Initialized, Role: FOLLOWER
I20260809 00:36:40.384892 7857 consensus_queue.cc:260] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9ef30f632c4cb4ac480c91de266a6b" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35747 } } peers { permanent_uuid: "807f10cef6a44d399b777185608b3f36" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 35409 } } peers { permanent_uuid: "7bee1f94fc354faeabebd6bd1d917829" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38991 } }
I20260809 00:36:40.385067 7858 tablet_bootstrap.cc:492] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829: No bootstrap required, opened a new log
I20260809 00:36:40.385210 7857 ts_tablet_manager.cc:1428] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:36:40.385145 7858 ts_tablet_manager.cc:1397] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 00:36:40.385293 7571 heartbeater.cc:503] Master 127.21.41.254:44169 was elected leader, sending a full tablet report...
I20260809 00:36:40.386989 7858 raft_consensus.cc:348] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9ef30f632c4cb4ac480c91de266a6b" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35747 } } peers { permanent_uuid: "807f10cef6a44d399b777185608b3f36" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 35409 } } peers { permanent_uuid: "7bee1f94fc354faeabebd6bd1d917829" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38991 } }
I20260809 00:36:40.387121 7858 raft_consensus.cc:374] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:40.387144 7858 raft_consensus.cc:729] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bee1f94fc354faeabebd6bd1d917829, State: Initialized, Role: FOLLOWER
I20260809 00:36:40.387221 7858 consensus_queue.cc:260] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9ef30f632c4cb4ac480c91de266a6b" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35747 } } peers { permanent_uuid: "807f10cef6a44d399b777185608b3f36" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 35409 } } peers { permanent_uuid: "7bee1f94fc354faeabebd6bd1d917829" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38991 } }
I20260809 00:36:40.387446 7858 ts_tablet_manager.cc:1428] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:36:40.387553 7839 heartbeater.cc:503] Master 127.21.41.254:44169 was elected leader, sending a full tablet report...
W20260809 00:36:40.416131 7706 tablet.cc:2369] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:36:40.508082 7572 tablet.cc:2369] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:36:40.527801 7862 raft_consensus.cc:482] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:36:40.527935 7862 raft_consensus.cc:504] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9ef30f632c4cb4ac480c91de266a6b" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35747 } } peers { permanent_uuid: "807f10cef6a44d399b777185608b3f36" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 35409 } } peers { permanent_uuid: "7bee1f94fc354faeabebd6bd1d917829" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38991 } }
I20260809 00:36:40.528258 7862 leader_election.cc:288] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af9ef30f632c4cb4ac480c91de266a6b (127.21.41.193:35747), 7bee1f94fc354faeabebd6bd1d917829 (127.21.41.195:38991)
I20260809 00:36:40.531193 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.528611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:45819 (local address 127.21.41.193:35747)
0809 00:36:40.528666 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.528669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.528679 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:36:40.528710 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.528711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.528724 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.528803 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:40.528971 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.529455 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.530451 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.530811 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.530913 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.530922 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.531088 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.531124 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:36:40.531193 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.528494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:35747 (local address 127.21.41.194:45819)
0809 00:36:40.528565 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:40.528580 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:40.528650 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:40.528808 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:40.528809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:40.528810 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:40.528890 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.528897 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.529545 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.529546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.530305 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.530307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.530910 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.530911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.530999 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.531002 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:36:40.531078 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 00:36:40.531118 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 00:36:40.531635 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.528597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:56921 (local address 127.21.41.195:38991)
0809 00:36:40.528677 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.528679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.528929 (+ 250us) server_negotiation.cc:407] Connection header received
0809 00:36:40.529023 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.529024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.529036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.529118 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:40.529307 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.529863 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.530860 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.531293 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.531370 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.531378 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.531536 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.531577 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:36:40.531639 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.528494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:38991 (local address 127.21.41.194:56921)
0809 00:36:40.528728 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:40.528744 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:40.528952 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:40.529147 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:40.529148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:40.529150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:40.529230 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.529231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.529959 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.529960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.530772 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.530773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.531367 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.531368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.531442 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.531446 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:36:40.531530 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:36:40.531565 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":56,"thread_start_us":108,"threads_started":1}
I20260809 00:36:40.531592 7524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0336c6949b4293832c1eb35f8192e4" candidate_uuid: "807f10cef6a44d399b777185608b3f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af9ef30f632c4cb4ac480c91de266a6b" is_pre_election: true
I20260809 00:36:40.531783 7524 raft_consensus.cc:2454] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 807f10cef6a44d399b777185608b3f36 in term 0.
I20260809 00:36:40.531930 7793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0336c6949b4293832c1eb35f8192e4" candidate_uuid: "807f10cef6a44d399b777185608b3f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bee1f94fc354faeabebd6bd1d917829" is_pre_election: true
I20260809 00:36:40.532037 7592 leader_election.cc:302] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 807f10cef6a44d399b777185608b3f36, af9ef30f632c4cb4ac480c91de266a6b; no voters:
I20260809 00:36:40.532115 7793 raft_consensus.cc:2454] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 807f10cef6a44d399b777185608b3f36 in term 0.
I20260809 00:36:40.532205 7862 raft_consensus.cc:2781] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:36:40.532315 7862 raft_consensus.cc:482] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:36:40.532352 7862 raft_consensus.cc:3037] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:40.533071 7862 raft_consensus.cc:504] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9ef30f632c4cb4ac480c91de266a6b" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35747 } } peers { permanent_uuid: "807f10cef6a44d399b777185608b3f36" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 35409 } } peers { permanent_uuid: "7bee1f94fc354faeabebd6bd1d917829" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38991 } }
I20260809 00:36:40.533241 7862 leader_election.cc:288] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [CANDIDATE]: Term 1 election: Requested vote from peers af9ef30f632c4cb4ac480c91de266a6b (127.21.41.193:35747), 7bee1f94fc354faeabebd6bd1d917829 (127.21.41.195:38991)
I20260809 00:36:40.533438 7524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0336c6949b4293832c1eb35f8192e4" candidate_uuid: "807f10cef6a44d399b777185608b3f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af9ef30f632c4cb4ac480c91de266a6b"
I20260809 00:36:40.533468 7793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0336c6949b4293832c1eb35f8192e4" candidate_uuid: "807f10cef6a44d399b777185608b3f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bee1f94fc354faeabebd6bd1d917829"
I20260809 00:36:40.533530 7793 raft_consensus.cc:3037] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:40.533531 7524 raft_consensus.cc:3037] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:40.534533 7793 raft_consensus.cc:2454] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 807f10cef6a44d399b777185608b3f36 in term 1.
I20260809 00:36:40.534533 7524 raft_consensus.cc:2454] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 807f10cef6a44d399b777185608b3f36 in term 1.
I20260809 00:36:40.534776 7595 leader_election.cc:302] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bee1f94fc354faeabebd6bd1d917829, 807f10cef6a44d399b777185608b3f36; no voters:
I20260809 00:36:40.534998 7862 raft_consensus.cc:2781] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:40.535166 7862 raft_consensus.cc:686] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 1 LEADER]: Becoming Leader. State: Replica: 807f10cef6a44d399b777185608b3f36, State: Running, Role: LEADER
I20260809 00:36:40.535279 7862 consensus_queue.cc:237] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9ef30f632c4cb4ac480c91de266a6b" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35747 } } peers { permanent_uuid: "807f10cef6a44d399b777185608b3f36" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 35409 } } peers { permanent_uuid: "7bee1f94fc354faeabebd6bd1d917829" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38991 } }
I20260809 00:36:40.536228 7378 catalog_manager.cc:5495] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 807f10cef6a44d399b777185608b3f36 (127.21.41.194). New cstate: current_term: 1 leader_uuid: "807f10cef6a44d399b777185608b3f36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9ef30f632c4cb4ac480c91de266a6b" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "807f10cef6a44d399b777185608b3f36" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bee1f94fc354faeabebd6bd1d917829" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260809 00:36:40.557215 7855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.554964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39390 (local address 127.21.41.194:35409)
0809 00:36:40.555030 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.555032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.555039 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:36:40.555075 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.555076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.555093 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.555156 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:36:40.555292 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.555763 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.556494 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.556691 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.556738 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.556849 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:36:40.556936 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:36:40.557064 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.557120 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.557132 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:36:40.559355 7793 raft_consensus.cc:1261] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829 [term 1 FOLLOWER]: Refusing update from remote peer 807f10cef6a44d399b777185608b3f36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:40.559370 7524 raft_consensus.cc:1261] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b [term 1 FOLLOWER]: Refusing update from remote peer 807f10cef6a44d399b777185608b3f36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:40.559870 7866 consensus_queue.cc:1035] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bee1f94fc354faeabebd6bd1d917829" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:40.560006 7862 consensus_queue.cc:1035] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af9ef30f632c4cb4ac480c91de266a6b" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 35747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:40.563491 7870 mvcc.cc:205] Tried to move back new op lower bound from 7316421839087955968 to 7316421838993219584. Current Snapshot: MvccSnapshot[applied={T|T < 7316421839087955968}]
I20260809 00:36:40.565434 7379 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:40.565543 7379 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:40.566587 7379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8370bed9e33d4118b0707ae9da15666a: Sending DeleteTablet for 3 replicas of tablet 2a0336c6949b4293832c1eb35f8192e4
I20260809 00:36:40.566963 7773 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a0336c6949b4293832c1eb35f8192e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:40 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:37760
I20260809 00:36:40.567046 7639 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a0336c6949b4293832c1eb35f8192e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:40 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:39376
I20260809 00:36:40.567065 7504 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a0336c6949b4293832c1eb35f8192e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:40 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:50312
I20260809 00:36:40.567335 7880 tablet_replica.cc:330] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b: stopping tablet replica
I20260809 00:36:40.567425 7879 tablet_replica.cc:330] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: stopping tablet replica
I20260809 00:36:40.567456 7880 raft_consensus.cc:2229] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:40.567566 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 7441
I20260809 00:36:40.567586 7880 raft_consensus.cc:2258] T 2a0336c6949b4293832c1eb35f8192e4 P af9ef30f632c4cb4ac480c91de266a6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:40.567595 7879 raft_consensus.cc:2229] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:36:40.567786 7879 raft_consensus.cc:2258] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:40.568524 7879 ts_tablet_manager.cc:1905] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:36:40.570236 7879 ts_tablet_manager.cc:1918] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:40.570375 7879 log.cc:1196] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/2a0336c6949b4293832c1eb35f8192e4
I20260809 00:36:40.570634 7879 ts_tablet_manager.cc:1939] T 2a0336c6949b4293832c1eb35f8192e4 P 807f10cef6a44d399b777185608b3f36: Deleting consensus metadata
I20260809 00:36:40.571084 7363 catalog_manager.cc:4841] TS 807f10cef6a44d399b777185608b3f36 (127.21.41.194:35409): tablet 2a0336c6949b4293832c1eb35f8192e4 (table test-table [id=0f870bb3e700478b9e269a1b304d4b58]) successfully deleted
I20260809 00:36:40.575026 7878 tablet_replica.cc:330] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829: stopping tablet replica
I20260809 00:36:40.575196 7878 raft_consensus.cc:2229] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:40.575328 7878 raft_consensus.cc:2258] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:40.575834 7878 ts_tablet_manager.cc:1905] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:36:40.577313 7363 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:35747 (error 108)
I20260809 00:36:40.577476 7878 ts_tablet_manager.cc:1918] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:40.577543 7878 log.cc:1196] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/2a0336c6949b4293832c1eb35f8192e4
I20260809 00:36:40.577551 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 7576
I20260809 00:36:40.578651 7878 ts_tablet_manager.cc:1939] T 2a0336c6949b4293832c1eb35f8192e4 P 7bee1f94fc354faeabebd6bd1d917829: Deleting consensus metadata
I20260809 00:36:40.578951 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.578803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:35747 (local address 127.0.0.1:50326)
0809 00:36:40.578865 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:40.578908 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 00:36:40.579116 7366 catalog_manager.cc:4841] TS 7bee1f94fc354faeabebd6bd1d917829 (127.21.41.195:38991): tablet 2a0336c6949b4293832c1eb35f8192e4 (table test-table [id=0f870bb3e700478b9e269a1b304d4b58]) successfully deleted
W20260809 00:36:40.579241 7363 catalog_manager.cc:4584] TS af9ef30f632c4cb4ac480c91de266a6b (127.21.41.193:35747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2a0336c6949b4293832c1eb35f8192e4: Network error: Client connection negotiation failed: client connection to 127.21.41.193:35747: connect: Connection refused (error 111)
I20260809 00:36:40.583264 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 7710
I20260809 00:36:40.588007 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 7347
2026-08-09T00:36:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2090 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T00:36:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:40Z Disabled control of system clock
I20260809 00:36:40.616554 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35247
--webserver_interface=127.21.41.254
--webserver_port=0
--builtin_ntp_servers=127.21.41.212:39789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:35247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:36:40.698225 7885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:40.698542 7885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:40.698568 7885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:40.699694 7885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:40.699728 7885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:40.699740 7885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:40.699752 7885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:40.699764 7885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:40.701035 7885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:35247
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:35247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:40.701249 7885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:40.701496 7885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:40.706313 7890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:40.706310 7891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:40.706449 7893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:40.706794 7885 server_base.cc:1034] running on GCE node
I20260809 00:36:40.707018 7885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:40.707255 7885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:40.708398 7885 hybrid_clock.cc:648] HybridClock initialized: now 1786235800708379 us; error 45 us; skew 500 ppm
I20260809 00:36:40.709702 7885 webserver.cc:460] Webserver started at http://127.21.41.254:34855/ using document root <none> and password file <none>
I20260809 00:36:40.709967 7885 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:40.710016 7885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:40.710177 7885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:40.711302 7885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "6a6bc8d88872414f841185d8a34aa25f"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "d238b34b285b2fb2b10ef593e389e872"
server_key_iv: "3a25c9c7fff6c72b13c120c12f21abd8"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.711684 7885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "6a6bc8d88872414f841185d8a34aa25f"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "d238b34b285b2fb2b10ef593e389e872"
server_key_iv: "3a25c9c7fff6c72b13c120c12f21abd8"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.713017 7885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:36:40.713878 7899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:40.714109 7885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:40.714218 7885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "6a6bc8d88872414f841185d8a34aa25f"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "d238b34b285b2fb2b10ef593e389e872"
server_key_iv: "3a25c9c7fff6c72b13c120c12f21abd8"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.714308 7885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:40.724135 7885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:40.724488 7885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:40.724650 7885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:40.729025 7885 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:35247
I20260809 00:36:40.729061 7951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:35247 every 8 connection(s)
I20260809 00:36:40.729414 7885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:40.730461 7952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:40.731182 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 7885
I20260809 00:36:40.731299 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:40.731498 21671 external_mini_cluster.cc:1428] Setting key f8129961027105989b24dfb9c9a3c258
I20260809 00:36:40.733704 7952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f: Bootstrap starting.
I20260809 00:36:40.734427 7952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:40.734735 7952 log.cc:824] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:40.735550 7952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f: No bootstrap required, opened a new log
I20260809 00:36:40.738008 7952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bc8d88872414f841185d8a34aa25f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35247 } }
I20260809 00:36:40.738180 7952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:40.738216 7952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6bc8d88872414f841185d8a34aa25f, State: Initialized, Role: FOLLOWER
I20260809 00:36:40.738251 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.732054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37052 (local address 127.21.41.254:35247)
0809 00:36:40.732284 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.732293 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.732326 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:36:40.732878 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.732887 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.732970 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.733120 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:40.733290 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.734634 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.735945 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.736568 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.736697 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.737433 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:40.737437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:40.737438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:40.737706 (+ 268us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:40.737756 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:40.737772 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:40.737853 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.737914 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.738044 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260809 00:36:40.738394 7952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bc8d88872414f841185d8a34aa25f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35247 } }
I20260809 00:36:40.738488 7952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:40.738528 7952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:40.738579 7952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:40.739224 7952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bc8d88872414f841185d8a34aa25f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35247 } }
I20260809 00:36:40.739316 7952 leader_election.cc:302] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a6bc8d88872414f841185d8a34aa25f; no voters:
I20260809 00:36:40.739449 7952 leader_election.cc:288] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:40.739517 7957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:40.739687 7957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [term 1 LEADER]: Becoming Leader. State: Replica: 6a6bc8d88872414f841185d8a34aa25f, State: Running, Role: LEADER
I20260809 00:36:40.739732 7952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:40.739771 7957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bc8d88872414f841185d8a34aa25f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35247 } }
I20260809 00:36:40.740126 7958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a6bc8d88872414f841185d8a34aa25f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bc8d88872414f841185d8a34aa25f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35247 } } }
I20260809 00:36:40.740134 7959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a6bc8d88872414f841185d8a34aa25f. Latest consensus state: current_term: 1 leader_uuid: "6a6bc8d88872414f841185d8a34aa25f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6bc8d88872414f841185d8a34aa25f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 35247 } } }
I20260809 00:36:40.740267 7959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:40.740428 7958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:40.740473 7965 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:40.741022 7965 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:40.742590 7965 catalog_manager.cc:1347] Generated new cluster ID: 8b389e7c510a433ba0d31f5d8d567e83
I20260809 00:36:40.742646 7965 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:40.754864 7965 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:40.755590 7965 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:40.761197 7965 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f: Generated new TSK 0
I20260809 00:36:40.761394 7965 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:40.768949 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35247
--builtin_ntp_servers=127.21.41.212:39789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:36:40.855700 7976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:40.855955 7976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:40.855996 7976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:40.857177 7976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:40.857247 7976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:40.857292 7976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:36:40.858920 7976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:40.859193 7976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:40.859519 7976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:40.865933 7981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:40.865962 7982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:40.866014 7984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:40.866518 7976 server_base.cc:1034] running on GCE node
I20260809 00:36:40.866737 7976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:40.867004 7976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:40.868175 7976 hybrid_clock.cc:648] HybridClock initialized: now 1786235800868160 us; error 27 us; skew 500 ppm
I20260809 00:36:40.869714 7976 webserver.cc:460] Webserver started at http://127.21.41.193:36765/ using document root <none> and password file <none>
I20260809 00:36:40.869982 7976 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:40.870049 7976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:40.870183 7976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:40.871789 7976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "c202bbb6ef9c4437852f39c0483fed74"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "a1c794359fd86616b51fee4044268b2e"
server_key_iv: "c02b45c941b2c51789a7486c827c5de2"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.872193 7976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "c202bbb6ef9c4437852f39c0483fed74"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "a1c794359fd86616b51fee4044268b2e"
server_key_iv: "c02b45c941b2c51789a7486c827c5de2"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.873821 7976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:36:40.874884 7990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:40.875070 7976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:36:40.875183 7976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "c202bbb6ef9c4437852f39c0483fed74"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "a1c794359fd86616b51fee4044268b2e"
server_key_iv: "c02b45c941b2c51789a7486c827c5de2"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.875272 7976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:40.883308 7976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:40.883628 7976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:40.883783 7976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:40.884028 7976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:40.884378 7976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:40.884414 7976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:40.884436 7976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:40.884449 7976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:40.890204 7976 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:43611
I20260809 00:36:40.890301 8103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:43611 every 8 connection(s)
I20260809 00:36:40.890672 7976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:36:40.894243 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 7976
I20260809 00:36:40.894331 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:36:40.894599 21671 external_mini_cluster.cc:1428] Setting key 8bedbe1fb5f24c3c9f35c46a6e0ca104
I20260809 00:36:40.895835 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35247
--builtin_ntp_servers=127.21.41.212:39789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 00:36:40.896540 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.891457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:38819 (local address 127.21.41.254:35247)
0809 00:36:40.891546 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:40.891548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:40.891748 (+ 200us) server_negotiation.cc:407] Connection header received
0809 00:36:40.892267 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:40.892268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:40.892283 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:40.892348 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:40.893054 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.893584 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.894815 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.895090 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.895180 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.896282 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:40.896284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:40.896284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:40.896289 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:40.896304 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:40.896309 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:40.896361 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:40.896430 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 00:36:40.896459 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 00:36:40.896783 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:40.891399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35247 (local address 127.21.41.193:38819)
0809 00:36:40.891624 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:40.891661 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:40.892153 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:40.892423 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:40.892426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:40.892435 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:40.892944 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.892948 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.893755 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.893756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.894728 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:40.894730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:40.895192 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:40.895193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:40.895336 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:40.896118 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:36:40.896118 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:40.896158 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:36:40.896182 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:40.896353 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:40.896359 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:40.896362 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:36:40.896425 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 00:36:40.896527 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":124,"threads_started":1}
I20260809 00:36:40.897518 8104 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35247
I20260809 00:36:40.897632 8104 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:40.897858 8104 heartbeater.cc:511] Master 127.21.41.254:35247 requested a full tablet report, sending...
I20260809 00:36:40.898454 7916 ts_manager.cc:194] Registered new tserver with Master: c202bbb6ef9c4437852f39c0483fed74 (127.21.41.193:43611)
I20260809 00:36:40.899257 7916 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.193:38819
W20260809 00:36:40.982597 8107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:40.982928 8107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:40.982954 8107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:40.984227 8107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:40.984270 8107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:40.984300 8107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:36:40.985880 8107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:40.986100 8107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:40.986425 8107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:40.991882 8115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:40.991940 8107 server_base.cc:1034] running on GCE node
W20260809 00:36:40.991884 8112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:40.991880 8113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:40.992360 8107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:40.992573 8107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:40.993714 8107 hybrid_clock.cc:648] HybridClock initialized: now 1786235800993694 us; error 40 us; skew 500 ppm
I20260809 00:36:40.995144 8107 webserver.cc:460] Webserver started at http://127.21.41.194:35747/ using document root <none> and password file <none>
I20260809 00:36:40.995414 8107 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:40.995466 8107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:40.995606 8107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:40.996743 8107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "8fec198f2bf94365aa9b3db35bf0c839"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "94fc43467743061eb31dd1d27d06e7f8"
server_key_iv: "4a79200176f2289538759072a2a56987"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.997278 8107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "8fec198f2bf94365aa9b3db35bf0c839"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "94fc43467743061eb31dd1d27d06e7f8"
server_key_iv: "4a79200176f2289538759072a2a56987"
server_key_version: "encryptionkey@0"
I20260809 00:36:40.999076 8107 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260809 00:36:41.000006 8121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:41.000255 8107 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:41.000371 8107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "8fec198f2bf94365aa9b3db35bf0c839"
format_stamp: "Formatted at 2026-08-09 00:36:40 on dist-test-slave-zpfg"
server_key: "94fc43467743061eb31dd1d27d06e7f8"
server_key_iv: "4a79200176f2289538759072a2a56987"
server_key_version: "encryptionkey@0"
I20260809 00:36:41.000455 8107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:41.026651 8107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:41.027030 8107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:41.027209 8107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:41.027526 8107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:41.027937 8107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:41.028005 8107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:41.028048 8107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:41.028076 8107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:41.034219 8107 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:44935
I20260809 00:36:41.034319 8234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:44935 every 8 connection(s)
I20260809 00:36:41.034719 8107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:36:41.040305 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.035536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:52651 (local address 127.21.41.254:35247)
0809 00:36:41.035614 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:41.035617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:41.035930 (+ 313us) server_negotiation.cc:407] Connection header received
0809 00:36:41.036450 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:41.036451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:41.036463 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:41.036517 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:41.037236 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.037746 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.038826 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.039017 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.039076 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.040042 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:41.040043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:41.040043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:41.040048 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:41.040061 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:41.040065 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:41.040110 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:41.040223 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 00:36:41.040244 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 00:36:41.040748 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.035525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35247 (local address 127.21.41.194:52651)
0809 00:36:41.035792 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:41.035829 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:41.036349 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:41.036582 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:41.036585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:41.036594 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:41.037115 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.037119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.037847 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.037848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.038740 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.038743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.039138 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.039139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.039286 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.039812 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:36:41.039813 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:41.039843 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:36:41.039869 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:41.040124 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:41.040129 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:41.040131 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:36:41.040222 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 00:36:41.040319 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":142,"threads_started":1}
I20260809 00:36:41.041241 8235 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35247
I20260809 00:36:41.041337 8235 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:41.041483 8235 heartbeater.cc:511] Master 127.21.41.254:35247 requested a full tablet report, sending...
I20260809 00:36:41.041862 7916 ts_manager.cc:194] Registered new tserver with Master: 8fec198f2bf94365aa9b3db35bf0c839 (127.21.41.194:44935)
I20260809 00:36:41.042222 7916 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.194:52651
I20260809 00:36:41.042990 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 8107
I20260809 00:36:41.043094 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:36:41.043296 21671 external_mini_cluster.cc:1428] Setting key bed6696c5d692c349937fbf8572ccdd2
I20260809 00:36:41.044237 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35247
--builtin_ntp_servers=127.21.41.212:39789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:36:41.127290 8238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:41.127512 8238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:41.127534 8238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:41.128755 8238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:41.128804 8238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:41.128830 8238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:36:41.130227 8238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:35247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:41.130537 8238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:41.130867 8238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:41.136073 8246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:41.136050 8244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:41.136413 8238 server_base.cc:1034] running on GCE node
W20260809 00:36:41.136054 8243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:41.136715 8238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:41.136940 8238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:41.138115 8238 hybrid_clock.cc:648] HybridClock initialized: now 1786235801138087 us; error 46 us; skew 500 ppm
I20260809 00:36:41.139670 8238 webserver.cc:460] Webserver started at http://127.21.41.195:35029/ using document root <none> and password file <none>
I20260809 00:36:41.139952 8238 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:41.140002 8238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:41.140101 8238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:41.141372 8238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "6c85c468d7754456b685c1dde6a6fb9b"
format_stamp: "Formatted at 2026-08-09 00:36:41 on dist-test-slave-zpfg"
server_key: "68e257453a05ca9b18cb974d16e6986e"
server_key_iv: "c26117d6d9ea5db8564f38acd72cb778"
server_key_version: "encryptionkey@0"
I20260809 00:36:41.141686 8238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "6c85c468d7754456b685c1dde6a6fb9b"
format_stamp: "Formatted at 2026-08-09 00:36:41 on dist-test-slave-zpfg"
server_key: "68e257453a05ca9b18cb974d16e6986e"
server_key_iv: "c26117d6d9ea5db8564f38acd72cb778"
server_key_version: "encryptionkey@0"
I20260809 00:36:41.143088 8238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 00:36:41.143812 8252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:41.144066 8238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:41.144203 8238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "6c85c468d7754456b685c1dde6a6fb9b"
format_stamp: "Formatted at 2026-08-09 00:36:41 on dist-test-slave-zpfg"
server_key: "68e257453a05ca9b18cb974d16e6986e"
server_key_iv: "c26117d6d9ea5db8564f38acd72cb778"
server_key_version: "encryptionkey@0"
I20260809 00:36:41.144280 8238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:41.180576 8238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:41.180927 8238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:41.181088 8238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:41.181354 8238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:41.181872 8238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:41.181908 8238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:41.181931 8238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:41.181984 8238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:41.187825 8238 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:34251
I20260809 00:36:41.187870 8365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:34251 every 8 connection(s)
I20260809 00:36:41.188289 8238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:36:41.189958 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 8238
I20260809 00:36:41.190075 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:36:41.190260 21671 external_mini_cluster.cc:1428] Setting key 42c87d6f102fe0b132e1bd673cccb244
I20260809 00:36:41.194181 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.189009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:36599 (local address 127.21.41.254:35247)
0809 00:36:41.189075 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:41.189077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:41.189301 (+ 224us) server_negotiation.cc:407] Connection header received
0809 00:36:41.189910 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:41.189912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:41.189927 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:41.190016 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:41.191139 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.191686 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.192691 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.192891 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.192963 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.193875 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:41.193876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:41.193877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:41.193882 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:41.193897 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:41.193901 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:41.193970 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:41.194071 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 00:36:41.194098 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:36:41.194447 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.188971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:35247 (local address 127.21.41.195:36599)
0809 00:36:41.189163 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:41.189204 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:41.189775 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:41.190131 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:41.190135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:41.190146 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:41.190915 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.190924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.191795 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.191796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.192595 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.192597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.193053 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.193054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.193188 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.193692 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:36:41.193693 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:41.193723 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:36:41.193746 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:41.193970 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:41.193977 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:41.193980 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:36:41.194076 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 00:36:41.194181 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260809 00:36:41.194900 8366 heartbeater.cc:347] Connected to a master server at 127.21.41.254:35247
I20260809 00:36:41.194991 8366 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:41.195137 8366 heartbeater.cc:511] Master 127.21.41.254:35247 requested a full tablet report, sending...
I20260809 00:36:41.195504 7916 ts_manager.cc:194] Registered new tserver with Master: 6c85c468d7754456b685c1dde6a6fb9b (127.21.41.195:34251)
I20260809 00:36:41.195868 7916 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.195:36599
I20260809 00:36:41.202239 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:36:41.207274 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.203860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37062 (local address 127.21.41.254:35247)
0809 00:36:41.203958 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:41.203960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:41.204091 (+ 131us) server_negotiation.cc:407] Connection header received
0809 00:36:41.204159 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:41.204160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:41.204170 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:41.204262 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:41.204436 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.205046 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.205888 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.206093 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.206169 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.206925 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:41.206926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:41.206926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:41.206931 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:41.206948 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:41.206952 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:41.207073 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:41.207173 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 00:36:41.207185 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 00:36:41.209229 7916 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:36:41.211010 7916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:36:41.217741 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.213368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45632 (local address 127.21.41.193:43611)
0809 00:36:41.213754 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:41.213759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:41.213770 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:36:41.213812 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:41.213814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:41.213830 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:41.213914 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:41.214178 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.215363 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.216705 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.217290 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.217374 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.217471 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:41.217567 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 00:36:41.217606 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":102,"threads_started":1}
I20260809 00:36:41.217761 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.213258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:34251 (local address 127.0.0.1:33362)
0809 00:36:41.213627 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:41.213638 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:41.213699 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:41.213954 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:41.213957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:41.213959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:41.214032 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.214033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.215116 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.215117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.216414 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.216417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.217408 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.217409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.217468 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.217489 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 00:36:41.217525 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 00:36:41.217606 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
I20260809 00:36:41.217698 8374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.213258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:43611 (local address 127.0.0.1:45632)
0809 00:36:41.213443 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:41.213483 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:41.213548 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:41.214035 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:41.214036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:41.214039 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:41.214092 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.214094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.215457 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.215458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.216635 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.216638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.217383 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.217384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.217464 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.217477 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 00:36:41.217554 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:36:41.217602 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":36,"thread_start_us":74,"threads_started":1}
I20260809 00:36:41.217999 8376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.213368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33362 (local address 127.21.41.195:34251)
0809 00:36:41.213609 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:41.213614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:41.213777 (+ 163us) server_negotiation.cc:407] Connection header received
0809 00:36:41.213827 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:41.213829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:41.213847 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:41.213930 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:41.214115 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.215016 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.216518 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.217283 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.217724 (+ 441us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.217818 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:41.217845 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 00:36:41.217890 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":89,"threads_started":1}
I20260809 00:36:41.218070 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.213476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:44935 (local address 127.0.0.1:44834)
0809 00:36:41.214017 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:41.214027 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:41.214074 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:41.214269 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:41.214270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:41.214272 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:41.214314 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.214316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.215595 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.215596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.216574 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.216576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.217528 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.217530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.217580 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.217582 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:36:41.217628 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 00:36:41.217663 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260809 00:36:41.218309 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.213684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.21.41.194:44935)
0809 00:36:41.213864 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:41.213868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:41.214133 (+ 265us) server_negotiation.cc:407] Connection header received
0809 00:36:41.214178 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:41.214180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:41.214196 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:41.214272 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:41.214463 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.215510 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.216668 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.217343 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.217406 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.217494 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:41.217746 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 00:36:41.218222 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260809 00:36:41.218734 8300 tablet_service.cc:1467] Processing CreateTablet for tablet 385ac5bd036340c8a3da5c243205f485 (DEFAULT_TABLE table=test-table [id=6e134b98e68c4354a36513da14a6c921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:41.218737 8038 tablet_service.cc:1467] Processing CreateTablet for tablet 385ac5bd036340c8a3da5c243205f485 (DEFAULT_TABLE table=test-table [id=6e134b98e68c4354a36513da14a6c921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:41.218909 8169 tablet_service.cc:1467] Processing CreateTablet for tablet 385ac5bd036340c8a3da5c243205f485 (DEFAULT_TABLE table=test-table [id=6e134b98e68c4354a36513da14a6c921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:41.219098 8300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385ac5bd036340c8a3da5c243205f485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:41.219100 8038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385ac5bd036340c8a3da5c243205f485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:41.219214 8169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385ac5bd036340c8a3da5c243205f485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:41.221576 8380 tablet_bootstrap.cc:492] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: Bootstrap starting.
I20260809 00:36:41.222394 8381 tablet_bootstrap.cc:492] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74: Bootstrap starting.
I20260809 00:36:41.222559 8382 tablet_bootstrap.cc:492] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: Bootstrap starting.
I20260809 00:36:41.222774 8380 tablet_bootstrap.cc:654] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:41.223084 8381 tablet_bootstrap.cc:654] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:41.223110 8380 log.cc:824] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:41.223227 8382 tablet_bootstrap.cc:654] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:41.223419 8381 log.cc:824] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:41.223568 8382 log.cc:824] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:41.223989 8380 tablet_bootstrap.cc:492] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: No bootstrap required, opened a new log
I20260809 00:36:41.224064 8380 ts_tablet_manager.cc:1397] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:36:41.224233 8381 tablet_bootstrap.cc:492] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74: No bootstrap required, opened a new log
I20260809 00:36:41.224309 8381 ts_tablet_manager.cc:1397] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:36:41.224529 8382 tablet_bootstrap.cc:492] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: No bootstrap required, opened a new log
I20260809 00:36:41.224613 8382 ts_tablet_manager.cc:1397] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:36:41.226320 8380 raft_consensus.cc:348] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c468d7754456b685c1dde6a6fb9b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 34251 } } peers { permanent_uuid: "c202bbb6ef9c4437852f39c0483fed74" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 43611 } } peers { permanent_uuid: "8fec198f2bf94365aa9b3db35bf0c839" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 44935 } }
I20260809 00:36:41.226553 8380 raft_consensus.cc:374] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:41.226600 8380 raft_consensus.cc:729] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c85c468d7754456b685c1dde6a6fb9b, State: Initialized, Role: FOLLOWER
I20260809 00:36:41.226733 8380 consensus_queue.cc:260] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c468d7754456b685c1dde6a6fb9b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 34251 } } peers { permanent_uuid: "c202bbb6ef9c4437852f39c0483fed74" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 43611 } } peers { permanent_uuid: "8fec198f2bf94365aa9b3db35bf0c839" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 44935 } }
I20260809 00:36:41.227041 8366 heartbeater.cc:503] Master 127.21.41.254:35247 was elected leader, sending a full tablet report...
I20260809 00:36:41.226949 8381 raft_consensus.cc:348] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c468d7754456b685c1dde6a6fb9b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 34251 } } peers { permanent_uuid: "c202bbb6ef9c4437852f39c0483fed74" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 43611 } } peers { permanent_uuid: "8fec198f2bf94365aa9b3db35bf0c839" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 44935 } }
I20260809 00:36:41.227097 8381 raft_consensus.cc:374] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:41.227139 8381 raft_consensus.cc:729] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c202bbb6ef9c4437852f39c0483fed74, State: Initialized, Role: FOLLOWER
I20260809 00:36:41.227262 8381 consensus_queue.cc:260] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c468d7754456b685c1dde6a6fb9b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 34251 } } peers { permanent_uuid: "c202bbb6ef9c4437852f39c0483fed74" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 43611 } } peers { permanent_uuid: "8fec198f2bf94365aa9b3db35bf0c839" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 44935 } }
I20260809 00:36:41.227221 8382 raft_consensus.cc:348] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c468d7754456b685c1dde6a6fb9b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 34251 } } peers { permanent_uuid: "c202bbb6ef9c4437852f39c0483fed74" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 43611 } } peers { permanent_uuid: "8fec198f2bf94365aa9b3db35bf0c839" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 44935 } }
I20260809 00:36:41.227026 8380 ts_tablet_manager.cc:1428] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:36:41.227355 8382 raft_consensus.cc:374] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:41.227392 8382 raft_consensus.cc:729] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fec198f2bf94365aa9b3db35bf0c839, State: Initialized, Role: FOLLOWER
I20260809 00:36:41.227509 8382 consensus_queue.cc:260] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c468d7754456b685c1dde6a6fb9b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 34251 } } peers { permanent_uuid: "c202bbb6ef9c4437852f39c0483fed74" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 43611 } } peers { permanent_uuid: "8fec198f2bf94365aa9b3db35bf0c839" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 44935 } }
I20260809 00:36:41.227553 8381 ts_tablet_manager.cc:1428] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 00:36:41.227679 8104 heartbeater.cc:503] Master 127.21.41.254:35247 was elected leader, sending a full tablet report...
I20260809 00:36:41.227756 8382 ts_tablet_manager.cc:1428] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:36:41.228039 8235 heartbeater.cc:503] Master 127.21.41.254:35247 was elected leader, sending a full tablet report...
W20260809 00:36:41.285339 8236 tablet.cc:2369] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:36:41.391396 8105 tablet.cc:2369] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:36:41.397540 8386 raft_consensus.cc:482] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:36:41.397652 8386 raft_consensus.cc:504] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c468d7754456b685c1dde6a6fb9b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 34251 } } peers { permanent_uuid: "c202bbb6ef9c4437852f39c0483fed74" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 43611 } } peers { permanent_uuid: "8fec198f2bf94365aa9b3db35bf0c839" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 44935 } }
I20260809 00:36:41.398069 8386 leader_election.cc:288] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c202bbb6ef9c4437852f39c0483fed74 (127.21.41.193:43611), 8fec198f2bf94365aa9b3db35bf0c839 (127.21.41.194:44935)
I20260809 00:36:41.401124 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.398478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:43611 (local address 127.21.41.195:38933)
0809 00:36:41.398548 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:41.398563 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:41.398615 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:41.398840 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:41.398841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:41.398842 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:41.398907 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.398909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.399541 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.399542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.400279 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.400281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.400853 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.400853 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.400930 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.400933 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:36:41.401007 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:36:41.401048 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 00:36:41.401124 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.398545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:38933 (local address 127.21.41.193:43611)
0809 00:36:41.398593 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:41.398596 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:41.398607 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:36:41.398718 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:41.398719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:41.398733 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:41.398784 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:41.398983 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.399457 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.400395 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.400759 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.400845 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.400863 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:41.401016 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 00:36:41.401056 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:36:41.401419 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.398730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:44813 (local address 127.21.41.194:44935)
0809 00:36:41.398792 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:41.398795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:41.398867 (+ 72us) server_negotiation.cc:407] Connection header received
0809 00:36:41.398930 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:41.398931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:41.398944 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:41.399025 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:41.399200 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.399733 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.400673 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.401031 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.401122 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.401131 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:41.401318 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 00:36:41.401367 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:36:41.401453 8058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385ac5bd036340c8a3da5c243205f485" candidate_uuid: "6c85c468d7754456b685c1dde6a6fb9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c202bbb6ef9c4437852f39c0483fed74" is_pre_election: true
I20260809 00:36:41.401643 8058 raft_consensus.cc:2454] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c85c468d7754456b685c1dde6a6fb9b in term 0.
I20260809 00:36:41.401661 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.398599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:44935 (local address 127.21.41.195:44813)
0809 00:36:41.398787 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:41.398797 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:41.398862 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:41.399029 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:41.399031 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:41.399032 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:41.399109 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.399111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.399835 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.399836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.400575 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:41.400577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.401144 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.401145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:41.401218 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.401223 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:36:41.401528 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 00:36:41.401575 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
I20260809 00:36:41.401902 8255 leader_election.cc:302] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c85c468d7754456b685c1dde6a6fb9b, c202bbb6ef9c4437852f39c0483fed74; no voters:
I20260809 00:36:41.402029 8386 raft_consensus.cc:2781] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:36:41.402087 8386 raft_consensus.cc:482] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:36:41.402108 8386 raft_consensus.cc:3037] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:41.402014 8189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385ac5bd036340c8a3da5c243205f485" candidate_uuid: "6c85c468d7754456b685c1dde6a6fb9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fec198f2bf94365aa9b3db35bf0c839" is_pre_election: true
I20260809 00:36:41.402196 8189 raft_consensus.cc:2454] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c85c468d7754456b685c1dde6a6fb9b in term 0.
I20260809 00:36:41.402992 8386 raft_consensus.cc:504] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c468d7754456b685c1dde6a6fb9b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 34251 } } peers { permanent_uuid: "c202bbb6ef9c4437852f39c0483fed74" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 43611 } } peers { permanent_uuid: "8fec198f2bf94365aa9b3db35bf0c839" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 44935 } }
I20260809 00:36:41.403122 8386 leader_election.cc:288] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [CANDIDATE]: Term 1 election: Requested vote from peers c202bbb6ef9c4437852f39c0483fed74 (127.21.41.193:43611), 8fec198f2bf94365aa9b3db35bf0c839 (127.21.41.194:44935)
I20260809 00:36:41.403311 8058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385ac5bd036340c8a3da5c243205f485" candidate_uuid: "6c85c468d7754456b685c1dde6a6fb9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c202bbb6ef9c4437852f39c0483fed74"
I20260809 00:36:41.403312 8189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385ac5bd036340c8a3da5c243205f485" candidate_uuid: "6c85c468d7754456b685c1dde6a6fb9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fec198f2bf94365aa9b3db35bf0c839"
I20260809 00:36:41.403389 8058 raft_consensus.cc:3037] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:41.403412 8189 raft_consensus.cc:3037] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:41.404213 8058 raft_consensus.cc:2454] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c85c468d7754456b685c1dde6a6fb9b in term 1.
I20260809 00:36:41.404330 8189 raft_consensus.cc:2454] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c85c468d7754456b685c1dde6a6fb9b in term 1.
I20260809 00:36:41.404501 8255 leader_election.cc:302] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c85c468d7754456b685c1dde6a6fb9b, 8fec198f2bf94365aa9b3db35bf0c839; no voters:
I20260809 00:36:41.404630 8386 raft_consensus.cc:2781] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:41.404870 8386 raft_consensus.cc:686] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 1 LEADER]: Becoming Leader. State: Replica: 6c85c468d7754456b685c1dde6a6fb9b, State: Running, Role: LEADER
I20260809 00:36:41.404994 8386 consensus_queue.cc:237] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c468d7754456b685c1dde6a6fb9b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 34251 } } peers { permanent_uuid: "c202bbb6ef9c4437852f39c0483fed74" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 43611 } } peers { permanent_uuid: "8fec198f2bf94365aa9b3db35bf0c839" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 44935 } }
I20260809 00:36:41.405967 7916 catalog_manager.cc:5495] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c85c468d7754456b685c1dde6a6fb9b (127.21.41.195). New cstate: current_term: 1 leader_uuid: "6c85c468d7754456b685c1dde6a6fb9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c468d7754456b685c1dde6a6fb9b" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 34251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c202bbb6ef9c4437852f39c0483fed74" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fec198f2bf94365aa9b3db35bf0c839" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 44935 } health_report { overall_health: UNKNOWN } } }
W20260809 00:36:41.438846 8367 tablet.cc:2369] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:36:41.448832 8376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.446077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.21.41.195:34251)
0809 00:36:41.446176 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:41.446178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:41.446189 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:36:41.446225 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:41.446226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:41.446246 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:41.446317 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:36:41.446528 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.447021 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.447956 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:41.448166 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:41.448250 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:41.448424 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:36:41.448519 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:36:41.448593 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:41.448689 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 00:36:41.448698 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 00:36:41.450878 8058 raft_consensus.cc:1261] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74 [term 1 FOLLOWER]: Refusing update from remote peer 6c85c468d7754456b685c1dde6a6fb9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:41.450892 8189 raft_consensus.cc:1261] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839 [term 1 FOLLOWER]: Refusing update from remote peer 6c85c468d7754456b685c1dde6a6fb9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:41.451290 8386 consensus_queue.cc:1035] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c202bbb6ef9c4437852f39c0483fed74" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:41.451426 8390 consensus_queue.cc:1035] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fec198f2bf94365aa9b3db35bf0c839" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 44935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:41.454382 8395 mvcc.cc:205] Tried to move back new op lower bound from 7316421842739675136 to 7316421842555609088. Current Snapshot: MvccSnapshot[applied={T|T < 7316421842739675136}]
I20260809 00:36:41.454542 8393 mvcc.cc:205] Tried to move back new op lower bound from 7316421842739675136 to 7316421842555609088. Current Snapshot: MvccSnapshot[applied={T|T < 7316421842739675136}]
I20260809 00:36:41.454994 8394 mvcc.cc:205] Tried to move back new op lower bound from 7316421842739675136 to 7316421842555609088. Current Snapshot: MvccSnapshot[applied={T|T < 7316421842739675136}]
I20260809 00:36:41.457079 7916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:41.457197 7916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:41.458004 7916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a6bc8d88872414f841185d8a34aa25f: Sending DeleteTablet for 3 replicas of tablet 385ac5bd036340c8a3da5c243205f485
I20260809 00:36:41.458433 8300 tablet_service.cc:1514] Processing DeleteTablet for tablet 385ac5bd036340c8a3da5c243205f485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:41 UTC) from {username='slave'} at 127.0.0.1:33362
I20260809 00:36:41.458433 8038 tablet_service.cc:1514] Processing DeleteTablet for tablet 385ac5bd036340c8a3da5c243205f485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:41 UTC) from {username='slave'} at 127.0.0.1:45632
I20260809 00:36:41.458454 8169 tablet_service.cc:1514] Processing DeleteTablet for tablet 385ac5bd036340c8a3da5c243205f485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:41 UTC) from {username='slave'} at 127.0.0.1:44834
I20260809 00:36:41.458681 8404 tablet_replica.cc:330] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: stopping tablet replica
I20260809 00:36:41.458798 8404 raft_consensus.cc:2229] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:36:41.458822 8403 tablet_replica.cc:330] T 385ac5bd036340c8a3da5c243205f485 P c202bbb6ef9c4437852f39c0483fed74: stopping tablet replica
I20260809 00:36:41.458950 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 7976
I20260809 00:36:41.459561 8404 raft_consensus.cc:2258] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:41.459760 8405 tablet_replica.cc:330] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: stopping tablet replica
I20260809 00:36:41.459875 8405 raft_consensus.cc:2229] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:41.460000 8405 raft_consensus.cc:2258] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:41.461237 8404 ts_tablet_manager.cc:1905] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:36:41.462108 8405 ts_tablet_manager.cc:1905] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:36:41.463498 8404 ts_tablet_manager.cc:1918] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:41.463583 8404 log.cc:1196] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/385ac5bd036340c8a3da5c243205f485
I20260809 00:36:41.463979 8404 ts_tablet_manager.cc:1939] T 385ac5bd036340c8a3da5c243205f485 P 6c85c468d7754456b685c1dde6a6fb9b: Deleting consensus metadata
I20260809 00:36:41.464283 8405 ts_tablet_manager.cc:1918] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:41.464424 8405 log.cc:1196] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/385ac5bd036340c8a3da5c243205f485
I20260809 00:36:41.464435 7901 catalog_manager.cc:4841] TS 6c85c468d7754456b685c1dde6a6fb9b (127.21.41.195:34251): tablet 385ac5bd036340c8a3da5c243205f485 (table test-table [id=6e134b98e68c4354a36513da14a6c921]) successfully deleted
I20260809 00:36:41.464758 8405 ts_tablet_manager.cc:1939] T 385ac5bd036340c8a3da5c243205f485 P 8fec198f2bf94365aa9b3db35bf0c839: Deleting consensus metadata
I20260809 00:36:41.465216 7902 catalog_manager.cc:4841] TS 8fec198f2bf94365aa9b3db35bf0c839 (127.21.41.194:44935): tablet 385ac5bd036340c8a3da5c243205f485 (table test-table [id=6e134b98e68c4354a36513da14a6c921]) successfully deleted
W20260809 00:36:41.467018 7902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:43611 (error 108)
I20260809 00:36:41.467195 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 8107
I20260809 00:36:41.468230 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:41.468090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:43611 (local address 127.0.0.1:45644)
0809 00:36:41.468150 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:41.468200 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:43611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 00:36:41.468453 7902 catalog_manager.cc:4584] TS c202bbb6ef9c4437852f39c0483fed74 (127.21.41.193:43611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 385ac5bd036340c8a3da5c243205f485: Network error: Client connection negotiation failed: client connection to 127.21.41.193:43611: connect: Connection refused (error 111)
I20260809 00:36:41.474004 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 8238
I20260809 00:36:41.479362 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 7885
2026-08-09T00:36:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (891 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:41 dist-test-slave-zpfg krb5kdc[8409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:41 dist-test-slave-zpfg krb5kdc[8409](info): set up 2 sockets
Aug 09 00:36:41 dist-test-slave-zpfg krb5kdc[8409](info): commencing operation
krb5kdc: starting...
W20260809 00:36:42.583827 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:42 dist-test-slave-zpfg krb5kdc[8409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:36:42.706307 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42931
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:42931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:42.792815 8422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:42.793035 8422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:42.793057 8422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:42.794251 8422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:42.794298 8422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:42.794312 8422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:42.794381 8422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:42.794407 8422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:42.795946 8422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:42931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:42.796219 8422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:42.796496 8422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:42.801411 8427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:42.801409 8428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:42.801383 8430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:42.801765 8422 server_base.cc:1034] running on GCE node
I20260809 00:36:42.801994 8422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:42.802239 8422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:42.803390 8422 hybrid_clock.cc:648] HybridClock initialized: now 1786235802803384 us; error 51 us; skew 500 ppm
Aug 09 00:36:42 dist-test-slave-zpfg krb5kdc[8409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:42.806172 8422 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:36:42.806712 8422 webserver.cc:460] Webserver started at http://127.21.41.254:44649/ using document root <none> and password file <none>
I20260809 00:36:42.806941 8422 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:42.807017 8422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:42.807184 8422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:42.808295 8422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "e34b96300ff549ef949b9aeb27c6db71"
format_stamp: "Formatted at 2026-08-09 00:36:42 on dist-test-slave-zpfg"
server_key: "ac216973b45d465819bb6feef12559ef"
server_key_iv: "ee794e45d8240cda18807295de414d73"
server_key_version: "encryptionkey@0"
I20260809 00:36:42.808672 8422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "e34b96300ff549ef949b9aeb27c6db71"
format_stamp: "Formatted at 2026-08-09 00:36:42 on dist-test-slave-zpfg"
server_key: "ac216973b45d465819bb6feef12559ef"
server_key_iv: "ee794e45d8240cda18807295de414d73"
server_key_version: "encryptionkey@0"
I20260809 00:36:42.810058 8422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:36:42.810983 8437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:42.811210 8422 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:42.811319 8422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "e34b96300ff549ef949b9aeb27c6db71"
format_stamp: "Formatted at 2026-08-09 00:36:42 on dist-test-slave-zpfg"
server_key: "ac216973b45d465819bb6feef12559ef"
server_key_iv: "ee794e45d8240cda18807295de414d73"
server_key_version: "encryptionkey@0"
I20260809 00:36:42.811417 8422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:42.823882 8422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:42.826175 8422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:42.826391 8422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:42.830189 8422 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:42931
I20260809 00:36:42.830238 8489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:42931 every 8 connection(s)
I20260809 00:36:42.830659 8422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:42.831226 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 8422
I20260809 00:36:42.831355 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:42.831578 21671 external_mini_cluster.cc:1428] Setting key 860b43599e776c72339145c4db0f73c5
I20260809 00:36:42.831596 8490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:42.834972 8490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71: Bootstrap starting.
I20260809 00:36:42.835708 8490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:42.835995 8490 log.cc:824] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:42.836803 8490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71: No bootstrap required, opened a new log
Aug 09 00:36:42 dist-test-slave-zpfg krb5kdc[8409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:42.839227 8490 raft_consensus.cc:348] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34b96300ff549ef949b9aeb27c6db71" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42931 } }
I20260809 00:36:42.839418 8490 raft_consensus.cc:374] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:42.839457 8490 raft_consensus.cc:729] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e34b96300ff549ef949b9aeb27c6db71, State: Initialized, Role: FOLLOWER
I20260809 00:36:42.839596 8490 consensus_queue.cc:260] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34b96300ff549ef949b9aeb27c6db71" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42931 } }
I20260809 00:36:42.839696 8490 raft_consensus.cc:388] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:42.839737 8490 raft_consensus.cc:482] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:42.839779 8490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:42.840461 8490 raft_consensus.cc:504] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34b96300ff549ef949b9aeb27c6db71" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42931 } }
I20260809 00:36:42.840575 8490 leader_election.cc:302] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e34b96300ff549ef949b9aeb27c6db71; no voters:
I20260809 00:36:42.840740 8490 leader_election.cc:288] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:42.840802 8495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:42.840988 8495 raft_consensus.cc:686] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [term 1 LEADER]: Becoming Leader. State: Replica: e34b96300ff549ef949b9aeb27c6db71, State: Running, Role: LEADER
I20260809 00:36:42.841007 8490 sys_catalog.cc:564] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:42.841078 8495 consensus_queue.cc:237] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34b96300ff549ef949b9aeb27c6db71" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42931 } }
I20260809 00:36:42.841419 8495 sys_catalog.cc:455] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e34b96300ff549ef949b9aeb27c6db71. Latest consensus state: current_term: 1 leader_uuid: "e34b96300ff549ef949b9aeb27c6db71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34b96300ff549ef949b9aeb27c6db71" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42931 } } }
I20260809 00:36:42.841514 8495 sys_catalog.cc:458] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:42.841430 8496 sys_catalog.cc:455] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e34b96300ff549ef949b9aeb27c6db71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34b96300ff549ef949b9aeb27c6db71" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42931 } } }
I20260809 00:36:42.841670 8496 sys_catalog.cc:458] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:42.842657 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:42.832104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48570 (local address 127.21.41.254:42931)
0809 00:36:42.832280 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:42.832288 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:42.832312 (+ 24us) server_negotiation.cc:407] Connection header received
0809 00:36:42.832732 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:42.832739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:42.832783 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:42.832896 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:42.833674 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:42.834623 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:42.835658 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:42.836116 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:42.836225 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:42.838758 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:42.838764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:42.838764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:42.838788 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:42.840823 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:42.841270 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:42.841273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:42.841273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:42.841374 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:42.841610 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:42.841611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:42.841611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:42.841950 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:42.842130 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:42.842260 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 00:36:42.842432 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
W20260809 00:36:42.842967 8511 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:36:42.843037 8511 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:36:42.843346 8502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:42.843521 8502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:42.844951 8502 catalog_manager.cc:1347] Generated new cluster ID: 02007c450475463c9231ac3c6d9d5521
I20260809 00:36:42.845038 8502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:42.856238 8502 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:42.856774 8502 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:42.862572 8502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71: Generated new TSK 0
I20260809 00:36:42.862828 8502 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.41.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
Entry for principal kudu/127.21.41.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab.
I20260809 00:36:42.893299 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42931
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:43.007261 8516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:43.007490 8516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:43.007520 8516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:43.008877 8516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:43.008931 8516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:43.008965 8516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:36:43.010775 8516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.193.keytab
--principal=kudu/127.21.41.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:43.011067 8516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:43.011374 8516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:43.017161 8524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:43.017203 8521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:43.017191 8522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:43.017409 8516 server_base.cc:1034] running on GCE node
I20260809 00:36:43.017649 8516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:43.017930 8516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:43.019096 8516 hybrid_clock.cc:648] HybridClock initialized: now 1786235803019098 us; error 46 us; skew 500 ppm
Aug 09 00:36:43 dist-test-slave-zpfg krb5kdc[8409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235803, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:43.028221 8516 init.cc:377] Logged in from keytab as kudu/127.21.41.193@KRBTEST.COM (short username kudu)
I20260809 00:36:43.028661 8516 webserver.cc:460] Webserver started at http://127.21.41.193:45377/ using document root <none> and password file <none>
I20260809 00:36:43.028930 8516 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:43.028980 8516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:43.029151 8516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:43.030296 8516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "a359b994142a434880eb9a1e43921486"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "e238b8ca0ae761dddfd4c11100547178"
server_key_iv: "415e058621efc4d938dc6e79e891b439"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.030722 8516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "a359b994142a434880eb9a1e43921486"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "e238b8ca0ae761dddfd4c11100547178"
server_key_iv: "415e058621efc4d938dc6e79e891b439"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.032162 8516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:43.032955 8531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.033135 8516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:43.033236 8516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "a359b994142a434880eb9a1e43921486"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "e238b8ca0ae761dddfd4c11100547178"
server_key_iv: "415e058621efc4d938dc6e79e891b439"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.033325 8516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:43.046901 8516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:43.049347 8516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:43.049520 8516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:43.049806 8516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:43.050135 8516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:43.050189 8516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.050230 8516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:43.050256 8516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.056326 8516 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:45537
I20260809 00:36:43.056373 8644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:45537 every 8 connection(s)
I20260809 00:36:43.056699 8516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:36:43.060020 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 8516
I20260809 00:36:43.060149 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:36:43.060365 21671 external_mini_cluster.cc:1428] Setting key c81292e020cd4bf7f5feeb3b2a7e5b52
Aug 09 00:36:43 dist-test-slave-zpfg krb5kdc[8409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235803, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.193@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:43.067462 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.057473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:37189 (local address 127.21.41.254:42931)
0809 00:36:43.057545 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.057547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.057731 (+ 184us) server_negotiation.cc:407] Connection header received
0809 00:36:43.058311 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.058312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.058327 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.058431 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:43.059452 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.059995 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.061425 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.061660 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.061751 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.064351 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:43.064353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:43.064353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:43.064363 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:43.065882 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:43.066331 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:43.066333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:43.066333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:43.066446 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:43.066726 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:43.066727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:43.066728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:43.066855 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:43.066936 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.067240 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.067359 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:36:43.067644 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.057421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42931 (local address 127.21.41.193:37189)
0809 00:36:43.057596 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:43.057640 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:43.058132 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:43.058507 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:43.058511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:43.059119 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:43.059331 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.059336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.060113 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.060114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.061331 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.061334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.061765 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.061766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.061907 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.062565 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:43.062574 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:43.064238 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:43.065992 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:43.066001 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:43.066003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:43.066244 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:43.066519 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:43.066521 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:43.066521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:43.066655 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:43.066927 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:43.066935 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:43.067092 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 00:36:43.067207 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 00:36:43.067385 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260809 00:36:43.068624 8645 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42931
I20260809 00:36:43.068746 8645 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.21.41.194@KRBTEST.COM; defaulting to no policy
I20260809 00:36:43.069006 8645 heartbeater.cc:511] Master 127.21.41.254:42931 requested a full tablet report, sending...
I20260809 00:36:43.069681 8454 ts_manager.cc:194] Registered new tserver with Master: a359b994142a434880eb9a1e43921486 (127.21.41.193:45537)
I20260809 00:36:43.070326 8454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.193@KRBTEST.COM'} at 127.21.41.193:37189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
Entry for principal kudu/127.21.41.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab.
I20260809 00:36:43.089926 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42931
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:43.190616 8650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:43.190850 8650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:43.190876 8650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:43.192262 8650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:43.192312 8650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:43.192343 8650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:36:43.193954 8650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.194.keytab
--principal=kudu/127.21.41.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:43.194229 8650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:43.194546 8650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:43.200404 8656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:43.200418 8658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:43.200418 8655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:43.200625 8650 server_base.cc:1034] running on GCE node
I20260809 00:36:43.200886 8650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:43.201135 8650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:43.202327 8650 hybrid_clock.cc:648] HybridClock initialized: now 1786235803202303 us; error 34 us; skew 500 ppm
Aug 09 00:36:43 dist-test-slave-zpfg krb5kdc[8409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235803, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:43.205178 8650 init.cc:377] Logged in from keytab as kudu/127.21.41.194@KRBTEST.COM (short username kudu)
I20260809 00:36:43.205569 8650 webserver.cc:460] Webserver started at http://127.21.41.194:45877/ using document root <none> and password file <none>
I20260809 00:36:43.205824 8650 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:43.205915 8650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:43.206071 8650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:43.207178 8650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "1a5a9718e7ce4850bc7a3c5514d29b43"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "f9a02612798dc1a411dee28ddcddbf6b"
server_key_iv: "7a59b7e3f743a8959db5edcbc16e5760"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.207547 8650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "1a5a9718e7ce4850bc7a3c5514d29b43"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "f9a02612798dc1a411dee28ddcddbf6b"
server_key_iv: "7a59b7e3f743a8959db5edcbc16e5760"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.208956 8650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:36:43.209753 8665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.210005 8650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:36:43.210114 8650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "1a5a9718e7ce4850bc7a3c5514d29b43"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "f9a02612798dc1a411dee28ddcddbf6b"
server_key_iv: "7a59b7e3f743a8959db5edcbc16e5760"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.210191 8650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:43.221554 8650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:43.223927 8650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:43.224102 8650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:43.224362 8650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:43.224700 8650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:43.224767 8650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.224834 8650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:43.224865 8650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.230855 8650 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:39377
I20260809 00:36:43.230929 8778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:39377 every 8 connection(s)
I20260809 00:36:43.231273 8650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:36:43.235433 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 8650
I20260809 00:36:43.235569 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:36:43.235818 21671 external_mini_cluster.cc:1428] Setting key d38a0c3853a7eb8e3bf4c8a7f6f79541
Aug 09 00:36:43 dist-test-slave-zpfg krb5kdc[8409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235803, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.194@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:43.242007 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.232065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:54525 (local address 127.21.41.254:42931)
0809 00:36:43.232131 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.232133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.232334 (+ 201us) server_negotiation.cc:407] Connection header received
0809 00:36:43.232898 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.232900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.232911 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.232968 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:43.233979 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.234585 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.235657 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.236036 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.236166 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.238631 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:43.238632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:43.238633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:43.238642 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:43.240403 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:43.240909 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:43.240911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:43.240911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:43.240975 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:43.241320 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:43.241321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:43.241321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:43.241450 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:43.241530 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.241819 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.241909 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:36:43.242300 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.232024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42931 (local address 127.21.41.194:54525)
0809 00:36:43.232188 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:43.232233 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:43.232786 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:43.233041 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:43.233045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:43.233664 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:43.233873 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.233877 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.234687 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.234688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.235531 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.235534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.236184 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.236185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.236334 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.236956 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:43.236964 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:43.238532 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:43.240531 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:43.240540 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:43.240542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:43.240793 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:43.241056 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:43.241057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:43.241057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:43.241187 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:43.241520 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:43.241527 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:43.241679 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 00:36:43.241798 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 00:36:43.241969 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260809 00:36:43.242904 8779 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42931
I20260809 00:36:43.243022 8779 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:43.243227 8779 heartbeater.cc:511] Master 127.21.41.254:42931 requested a full tablet report, sending...
I20260809 00:36:43.243628 8454 ts_manager.cc:194] Registered new tserver with Master: 1a5a9718e7ce4850bc7a3c5514d29b43 (127.21.41.194:39377)
I20260809 00:36:43.244038 8454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.194@KRBTEST.COM'} at 127.21.41.194:54525
WARNING: no policy specified for kudu/127.21.41.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
Entry for principal kudu/127.21.41.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab.
I20260809 00:36:43.263490 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42931
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:43.346552 8784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:43.346766 8784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:43.346792 8784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:43.348009 8784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:43.348057 8784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:43.348084 8784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:36:43.349604 8784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.195.keytab
--principal=kudu/127.21.41.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:42931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:43.349874 8784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:43.350117 8784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:43.355284 8790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:43.355255 8789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:43.355270 8792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:43.355552 8784 server_base.cc:1034] running on GCE node
I20260809 00:36:43.355794 8784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:43.356078 8784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:43.357272 8784 hybrid_clock.cc:648] HybridClock initialized: now 1786235803357263 us; error 35 us; skew 500 ppm
Aug 09 00:36:43 dist-test-slave-zpfg krb5kdc[8409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235803, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:43.359997 8784 init.cc:377] Logged in from keytab as kudu/127.21.41.195@KRBTEST.COM (short username kudu)
I20260809 00:36:43.360448 8784 webserver.cc:460] Webserver started at http://127.21.41.195:36053/ using document root <none> and password file <none>
I20260809 00:36:43.360652 8784 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:43.360693 8784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:43.360782 8784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:43.361781 8784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "d3baa68761ce4259b59406bba21351ff"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "fa577c62d29c8286aab4b015a33c6ac8"
server_key_iv: "72b54e6ba81cf9340e153dfc54c68504"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.362113 8784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "d3baa68761ce4259b59406bba21351ff"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "fa577c62d29c8286aab4b015a33c6ac8"
server_key_iv: "72b54e6ba81cf9340e153dfc54c68504"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.363454 8784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:36:43.364172 8799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.364368 8784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:36:43.364455 8784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "d3baa68761ce4259b59406bba21351ff"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "fa577c62d29c8286aab4b015a33c6ac8"
server_key_iv: "72b54e6ba81cf9340e153dfc54c68504"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.364511 8784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:43.372159 8784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:43.374614 8784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:43.374742 8784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:43.374938 8784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:43.375262 8784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:43.375299 8784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.375345 8784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:43.375372 8784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.381162 8784 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:38485
I20260809 00:36:43.381301 8912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:38485 every 8 connection(s)
I20260809 00:36:43.381606 8784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:36:43 dist-test-slave-zpfg krb5kdc[8409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235803, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.195@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:36:43.388598 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 8784
I20260809 00:36:43.388715 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:36:43.388942 21671 external_mini_cluster.cc:1428] Setting key d07d5648f8b6a8ac809e9a3f891640e2
I20260809 00:36:43.391898 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.382426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:57085 (local address 127.21.41.254:42931)
0809 00:36:43.382480 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.382482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.382769 (+ 287us) server_negotiation.cc:407] Connection header received
0809 00:36:43.383364 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.383366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.383378 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.383448 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:43.384518 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.385132 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.386169 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.386412 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.386466 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.388752 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:43.388753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:43.388754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:43.388764 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:43.390573 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:43.390990 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:43.390991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:43.390992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:43.391044 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:43.391283 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:43.391284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:43.391284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:43.391389 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:43.391439 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.391756 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.391832 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:36:43.392156 8915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.382399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:42931 (local address 127.21.41.195:57085)
0809 00:36:43.382577 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:43.382615 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:43.383226 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:43.383529 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:43.383533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:43.384189 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:43.384399 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.384404 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.385248 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.385249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.386085 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.386087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.386499 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.386501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.386624 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.387112 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:36:43.387121 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:43.388600 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:43.390676 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:43.390683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:43.390685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:43.390920 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:43.391105 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:36:43.391106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:36:43.391106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:36:43.391217 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:36:43.391443 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:43.391447 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:43.391564 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 00:36:43.391731 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 00:36:43.391870 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260809 00:36:43.392669 8913 heartbeater.cc:347] Connected to a master server at 127.21.41.254:42931
I20260809 00:36:43.392778 8913 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:43.392961 8913 heartbeater.cc:511] Master 127.21.41.254:42931 requested a full tablet report, sending...
I20260809 00:36:43.393378 8454 ts_manager.cc:194] Registered new tserver with Master: d3baa68761ce4259b59406bba21351ff (127.21.41.195:38485)
I20260809 00:36:43.393800 8454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.41.195@KRBTEST.COM'} at 127.21.41.195:57085
I20260809 00:36:43.401067 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:36:43.409058 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.402636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.21.41.254:42931)
0809 00:36:43.402680 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.402682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.402881 (+ 199us) server_negotiation.cc:407] Connection header received
0809 00:36:43.402962 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.402963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.402974 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.403033 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:43.403643 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.404161 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.404908 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.405123 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.405203 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.406496 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:43.406497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:43.406498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:43.406506 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:43.407808 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:43.408196 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:43.408198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:43.408198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:43.408259 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:43.408478 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:43.408479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:43.408479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:43.408644 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:43.408756 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.408859 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.408948 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:36:43.411067 8454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:36:43.412819 8454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:36:43.420149 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.415660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.21.41.193:45537)
0809 00:36:43.415877 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.415882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.415895 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:36:43.416096 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.416099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.416118 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.416210 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:43.416387 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.417524 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.419208 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.419746 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.419841 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.419897 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.419994 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.420038 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":103,"threads_started":1}
I20260809 00:36:43.420202 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.415435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:45537 (local address 127.0.0.1:38330)
0809 00:36:43.415757 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:43.415809 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:43.415889 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:43.416217 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:43.416220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:43.416223 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:43.416310 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.416312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.417651 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.417652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.419116 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.419118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.419845 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.419846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.419919 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.419923 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:36:43.419982 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:36:43.420060 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":93,"thread_start_us":66,"threads_started":1}
I20260809 00:36:43.420203 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.415438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:38485 (local address 127.0.0.1:43478)
0809 00:36:43.416136 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:43.416146 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:43.416199 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:43.416397 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:43.416398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:43.416400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:43.416451 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.416453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.417694 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.417695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.418822 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.418824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.419811 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.419812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.419899 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.419913 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:36:43.419987 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:36:43.420074 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":110,"spinlock_wait_cycles":768,"thread_start_us":114,"threads_started":1}
I20260809 00:36:43.420347 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.415717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56074 (local address 127.21.41.194:39377)
0809 00:36:43.415954 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.415957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.416620 (+ 663us) server_negotiation.cc:407] Connection header received
0809 00:36:43.416664 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.416666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.416678 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.416765 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:43.416868 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.417563 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.419109 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.419822 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.419889 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.419944 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.420202 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.420250 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":120,"threads_started":1}
I20260809 00:36:43.420558 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.415443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:39377 (local address 127.0.0.1:56074)
0809 00:36:43.416498 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:43.416508 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:43.416554 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:43.416762 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:43.416763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:43.416765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:43.416811 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.416813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.417695 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.417696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.419018 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.419021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.419975 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.419976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.420033 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.420037 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:36:43.420081 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 00:36:43.420117 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1035,"thread_start_us":116,"threads_started":1}
I20260809 00:36:43.421156 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.415880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43478 (local address 127.21.41.195:38485)
0809 00:36:43.416080 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.416085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.416258 (+ 173us) server_negotiation.cc:407] Connection header received
0809 00:36:43.416306 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.416308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.416325 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.416400 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:43.416544 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.417604 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.418913 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.419630 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.419702 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.419760 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.421050 (+ 1290us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.421087 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260809 00:36:43.421262 8713 tablet_service.cc:1467] Processing CreateTablet for tablet f90af7c2d7fa4b20a565ccac4d434304 (DEFAULT_TABLE table=test-table [id=66d0ecc40f364f9f99a96befd3454ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:43.421389 8579 tablet_service.cc:1467] Processing CreateTablet for tablet f90af7c2d7fa4b20a565ccac4d434304 (DEFAULT_TABLE table=test-table [id=66d0ecc40f364f9f99a96befd3454ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:43.421650 8713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90af7c2d7fa4b20a565ccac4d434304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:43.421657 8579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90af7c2d7fa4b20a565ccac4d434304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:43.421732 8847 tablet_service.cc:1467] Processing CreateTablet for tablet f90af7c2d7fa4b20a565ccac4d434304 (DEFAULT_TABLE table=test-table [id=66d0ecc40f364f9f99a96befd3454ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:43.421980 8847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90af7c2d7fa4b20a565ccac4d434304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:43.424525 8930 tablet_bootstrap.cc:492] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff: Bootstrap starting.
I20260809 00:36:43.424984 8931 tablet_bootstrap.cc:492] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486: Bootstrap starting.
I20260809 00:36:43.425566 8930 tablet_bootstrap.cc:654] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:43.425724 8931 tablet_bootstrap.cc:654] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:43.426271 8930 log.cc:824] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:43.426376 8931 log.cc:824] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:43.426520 8932 tablet_bootstrap.cc:492] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: Bootstrap starting.
I20260809 00:36:43.427095 8931 tablet_bootstrap.cc:492] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486: No bootstrap required, opened a new log
I20260809 00:36:43.427155 8931 ts_tablet_manager.cc:1397] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:36:43.427150 8930 tablet_bootstrap.cc:492] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff: No bootstrap required, opened a new log
I20260809 00:36:43.427208 8932 tablet_bootstrap.cc:654] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:43.427218 8930 ts_tablet_manager.cc:1397] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 00:36:43.427748 8932 log.cc:824] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:43.428632 8932 tablet_bootstrap.cc:492] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: No bootstrap required, opened a new log
I20260809 00:36:43.428700 8932 ts_tablet_manager.cc:1397] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:36:43.429121 8931 raft_consensus.cc:348] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a359b994142a434880eb9a1e43921486" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 45537 } } peers { permanent_uuid: "d3baa68761ce4259b59406bba21351ff" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38485 } } peers { permanent_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 39377 } }
I20260809 00:36:43.429325 8931 raft_consensus.cc:374] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:43.429373 8931 raft_consensus.cc:729] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a359b994142a434880eb9a1e43921486, State: Initialized, Role: FOLLOWER
I20260809 00:36:43.429363 8930 raft_consensus.cc:348] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a359b994142a434880eb9a1e43921486" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 45537 } } peers { permanent_uuid: "d3baa68761ce4259b59406bba21351ff" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38485 } } peers { permanent_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 39377 } }
I20260809 00:36:43.429494 8931 consensus_queue.cc:260] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a359b994142a434880eb9a1e43921486" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 45537 } } peers { permanent_uuid: "d3baa68761ce4259b59406bba21351ff" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38485 } } peers { permanent_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 39377 } }
I20260809 00:36:43.429540 8930 raft_consensus.cc:374] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:43.429579 8930 raft_consensus.cc:729] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3baa68761ce4259b59406bba21351ff, State: Initialized, Role: FOLLOWER
I20260809 00:36:43.429685 8930 consensus_queue.cc:260] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a359b994142a434880eb9a1e43921486" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 45537 } } peers { permanent_uuid: "d3baa68761ce4259b59406bba21351ff" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38485 } } peers { permanent_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 39377 } }
I20260809 00:36:43.429817 8931 ts_tablet_manager.cc:1428] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:36:43.429862 8645 heartbeater.cc:503] Master 127.21.41.254:42931 was elected leader, sending a full tablet report...
I20260809 00:36:43.429951 8930 ts_tablet_manager.cc:1428] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:36:43.430095 8913 heartbeater.cc:503] Master 127.21.41.254:42931 was elected leader, sending a full tablet report...
I20260809 00:36:43.430786 8932 raft_consensus.cc:348] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a359b994142a434880eb9a1e43921486" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 45537 } } peers { permanent_uuid: "d3baa68761ce4259b59406bba21351ff" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38485 } } peers { permanent_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 39377 } }
I20260809 00:36:43.430972 8932 raft_consensus.cc:374] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:43.431020 8932 raft_consensus.cc:729] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a5a9718e7ce4850bc7a3c5514d29b43, State: Initialized, Role: FOLLOWER
I20260809 00:36:43.431142 8932 consensus_queue.cc:260] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a359b994142a434880eb9a1e43921486" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 45537 } } peers { permanent_uuid: "d3baa68761ce4259b59406bba21351ff" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38485 } } peers { permanent_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 39377 } }
I20260809 00:36:43.431454 8779 heartbeater.cc:503] Master 127.21.41.254:42931 was elected leader, sending a full tablet report...
I20260809 00:36:43.431458 8932 ts_tablet_manager.cc:1428] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260809 00:36:43.481986 8780 tablet.cc:2369] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:36:43.541824 8937 raft_consensus.cc:482] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:36:43.542011 8937 raft_consensus.cc:504] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a359b994142a434880eb9a1e43921486" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 45537 } } peers { permanent_uuid: "d3baa68761ce4259b59406bba21351ff" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38485 } } peers { permanent_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 39377 } }
I20260809 00:36:43.542512 8937 leader_election.cc:288] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a359b994142a434880eb9a1e43921486 (127.21.41.193:45537), 1a5a9718e7ce4850bc7a3c5514d29b43 (127.21.41.194:39377)
I20260809 00:36:43.546837 8915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.542707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:39377 (local address 127.21.41.195:43245)
0809 00:36:43.542815 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:43.542827 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:43.542907 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:43.543156 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:43.543157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:43.543159 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:43.543231 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.543233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.544292 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.544294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.545571 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.545574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.546536 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.546537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.546628 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.546633 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:36:43.546703 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:36:43.546749 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":4}
I20260809 00:36:43.546895 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.542825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:43245 (local address 127.21.41.194:39377)
0809 00:36:43.542871 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.542873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.542883 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:36:43.543017 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.543018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.543033 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.543095 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:43.543386 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.544170 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.545676 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.546435 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.546543 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.546558 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.546724 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.546788 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:36:43.547256 8733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90af7c2d7fa4b20a565ccac4d434304" candidate_uuid: "d3baa68761ce4259b59406bba21351ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" is_pre_election: true
I20260809 00:36:43.547463 8733 raft_consensus.cc:2454] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3baa68761ce4259b59406bba21351ff in term 0.
I20260809 00:36:43.547731 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.542983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:60063 (local address 127.21.41.193:45537)
0809 00:36:43.543032 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.543035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.543125 (+ 90us) server_negotiation.cc:407] Connection header received
0809 00:36:43.543258 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.543259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.543273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.543340 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:43.543495 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.544312 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.546481 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.547104 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.547178 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.547186 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.547582 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.547666 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:36:43.547783 8802 leader_election.cc:302] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a5a9718e7ce4850bc7a3c5514d29b43, d3baa68761ce4259b59406bba21351ff; no voters:
I20260809 00:36:43.547921 8937 raft_consensus.cc:2781] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:36:43.547982 8937 raft_consensus.cc:482] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:36:43.548009 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.542712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:45537 (local address 127.21.41.195:60063)
0809 00:36:43.543061 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:43.543071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:43.543152 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:43.543339 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:43.543340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:43.543341 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:43.543398 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.543399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.544401 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.544402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.545570 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.545574 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.547368 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.547370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.547461 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.547466 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:36:43.547820 (+ 354us) client_negotiation.cc:240] Negotiation successful
0809 00:36:43.547892 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":52,"thread_start_us":120,"threads_started":1}
I20260809 00:36:43.548009 8937 raft_consensus.cc:3037] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:43.548506 8599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90af7c2d7fa4b20a565ccac4d434304" candidate_uuid: "d3baa68761ce4259b59406bba21351ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a359b994142a434880eb9a1e43921486" is_pre_election: true
I20260809 00:36:43.548681 8599 raft_consensus.cc:2454] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3baa68761ce4259b59406bba21351ff in term 0.
I20260809 00:36:43.549147 8937 raft_consensus.cc:504] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a359b994142a434880eb9a1e43921486" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 45537 } } peers { permanent_uuid: "d3baa68761ce4259b59406bba21351ff" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38485 } } peers { permanent_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 39377 } }
I20260809 00:36:43.549394 8937 leader_election.cc:288] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [CANDIDATE]: Term 1 election: Requested vote from peers a359b994142a434880eb9a1e43921486 (127.21.41.193:45537), 1a5a9718e7ce4850bc7a3c5514d29b43 (127.21.41.194:39377)
I20260809 00:36:43.549503 8733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90af7c2d7fa4b20a565ccac4d434304" candidate_uuid: "d3baa68761ce4259b59406bba21351ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43"
I20260809 00:36:43.549590 8733 raft_consensus.cc:3037] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:43.550737 8733 raft_consensus.cc:2454] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3baa68761ce4259b59406bba21351ff in term 1.
I20260809 00:36:43.550868 8599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90af7c2d7fa4b20a565ccac4d434304" candidate_uuid: "d3baa68761ce4259b59406bba21351ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a359b994142a434880eb9a1e43921486"
I20260809 00:36:43.550967 8802 leader_election.cc:302] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a5a9718e7ce4850bc7a3c5514d29b43, d3baa68761ce4259b59406bba21351ff; no voters:
I20260809 00:36:43.550953 8599 raft_consensus.cc:3037] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:43.551065 8937 raft_consensus.cc:2781] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:43.551203 8937 raft_consensus.cc:686] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 1 LEADER]: Becoming Leader. State: Replica: d3baa68761ce4259b59406bba21351ff, State: Running, Role: LEADER
I20260809 00:36:43.551294 8937 consensus_queue.cc:237] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a359b994142a434880eb9a1e43921486" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 45537 } } peers { permanent_uuid: "d3baa68761ce4259b59406bba21351ff" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38485 } } peers { permanent_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 39377 } }
I20260809 00:36:43.552078 8599 raft_consensus.cc:2454] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3baa68761ce4259b59406bba21351ff in term 1.
I20260809 00:36:43.552544 8454 catalog_manager.cc:5495] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff reported cstate change: term changed from 0 to 1, leader changed from <none> to d3baa68761ce4259b59406bba21351ff (127.21.41.195). New cstate: current_term: 1 leader_uuid: "d3baa68761ce4259b59406bba21351ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a359b994142a434880eb9a1e43921486" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 45537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3baa68761ce4259b59406bba21351ff" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 38485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 39377 } health_report { overall_health: UNKNOWN } } }
W20260809 00:36:43.557483 8646 tablet.cc:2369] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:36:43.567451 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.564590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43486 (local address 127.21.41.195:38485)
0809 00:36:43.564664 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.564667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.564678 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:36:43.564726 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.564727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.564750 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.564835 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:36:43.565010 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.565616 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.566516 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.566753 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.566849 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.567054 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:36:43.567178 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:36:43.567289 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.567311 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.567334 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:36:43.569437 8599 raft_consensus.cc:1261] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486 [term 1 FOLLOWER]: Refusing update from remote peer d3baa68761ce4259b59406bba21351ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:43.569448 8733 raft_consensus.cc:1261] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43 [term 1 FOLLOWER]: Refusing update from remote peer d3baa68761ce4259b59406bba21351ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:43.569886 8940 consensus_queue.cc:1035] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a359b994142a434880eb9a1e43921486" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 45537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:43.570043 8937 consensus_queue.cc:1035] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a5a9718e7ce4850bc7a3c5514d29b43" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:43.572822 8945 mvcc.cc:205] Tried to move back new op lower bound from 7316421851417608192 to 7316421851346845696. Current Snapshot: MvccSnapshot[applied={T|T < 7316421851417608192}]
I20260809 00:36:43.573750 8943 mvcc.cc:205] Tried to move back new op lower bound from 7316421851417608192 to 7316421851346845696. Current Snapshot: MvccSnapshot[applied={T|T < 7316421851417608192 or (T in {7316421851417608192})}]
I20260809 00:36:43.575348 8454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:43.575443 8454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:43.576333 8454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e34b96300ff549ef949b9aeb27c6db71: Sending DeleteTablet for 3 replicas of tablet f90af7c2d7fa4b20a565ccac4d434304
I20260809 00:36:43.576663 8579 tablet_service.cc:1514] Processing DeleteTablet for tablet f90af7c2d7fa4b20a565ccac4d434304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:43 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:38330
I20260809 00:36:43.576740 8847 tablet_service.cc:1514] Processing DeleteTablet for tablet f90af7c2d7fa4b20a565ccac4d434304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:43 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:43478
I20260809 00:36:43.576778 8713 tablet_service.cc:1514] Processing DeleteTablet for tablet f90af7c2d7fa4b20a565ccac4d434304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:43 UTC) from {username='kudu', principal='kudu/127.21.41.254@KRBTEST.COM'} at 127.0.0.1:56074
I20260809 00:36:43.576995 8953 tablet_replica.cc:330] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486: stopping tablet replica
I20260809 00:36:43.576998 8955 tablet_replica.cc:330] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: stopping tablet replica
I20260809 00:36:43.577098 8953 raft_consensus.cc:2229] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:43.577099 8955 raft_consensus.cc:2229] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:43.577203 8955 raft_consensus.cc:2258] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:43.577205 8953 raft_consensus.cc:2258] T f90af7c2d7fa4b20a565ccac4d434304 P a359b994142a434880eb9a1e43921486 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:43.577207 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 8516
I20260809 00:36:43.578229 8955 ts_tablet_manager.cc:1905] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:36:43.580178 8955 ts_tablet_manager.cc:1918] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:43.580250 8955 log.cc:1196] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/f90af7c2d7fa4b20a565ccac4d434304
I20260809 00:36:43.580535 8955 ts_tablet_manager.cc:1939] T f90af7c2d7fa4b20a565ccac4d434304 P 1a5a9718e7ce4850bc7a3c5514d29b43: Deleting consensus metadata
I20260809 00:36:43.580983 8440 catalog_manager.cc:4841] TS 1a5a9718e7ce4850bc7a3c5514d29b43 (127.21.41.194:39377): tablet f90af7c2d7fa4b20a565ccac4d434304 (table test-table [id=66d0ecc40f364f9f99a96befd3454ba4]) successfully deleted
I20260809 00:36:43.582952 8954 tablet_replica.cc:330] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff: stopping tablet replica
I20260809 00:36:43.583081 8954 raft_consensus.cc:2229] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:36:43.583205 8954 raft_consensus.cc:2258] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:43.583757 8954 ts_tablet_manager.cc:1905] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:36:43.584268 8438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:45537 (error 108)
I20260809 00:36:43.584920 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 8650
I20260809 00:36:43.585956 8954 ts_tablet_manager.cc:1918] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:43.586043 8954 log.cc:1196] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/f90af7c2d7fa4b20a565ccac4d434304
I20260809 00:36:43.586382 8954 ts_tablet_manager.cc:1939] T f90af7c2d7fa4b20a565ccac4d434304 P d3baa68761ce4259b59406bba21351ff: Deleting consensus metadata
I20260809 00:36:43.586872 8439 catalog_manager.cc:4841] TS d3baa68761ce4259b59406bba21351ff (127.21.41.195:38485): tablet f90af7c2d7fa4b20a565ccac4d434304 (table test-table [id=66d0ecc40f364f9f99a96befd3454ba4]) successfully deleted
I20260809 00:36:43.591840 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.591680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:45537 (local address 127.0.0.1:38342)
0809 00:36:43.591747 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:43.591798 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:36:43.591969 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 8784
W20260809 00:36:43.592001 8438 catalog_manager.cc:4584] TS a359b994142a434880eb9a1e43921486 (127.21.41.193:45537): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f90af7c2d7fa4b20a565ccac4d434304: Network error: Client connection negotiation failed: client connection to 127.21.41.193:45537: connect: Connection refused (error 111)
I20260809 00:36:43.597241 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 8422
2026-08-09T00:36:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2118 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T00:36:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:43Z Disabled control of system clock
I20260809 00:36:43.626318 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:37911
--webserver_interface=127.21.41.254
--webserver_port=0
--builtin_ntp_servers=127.21.41.212:42503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:37911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:36:43.708199 8960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:43.708422 8960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:43.708444 8960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:43.709518 8960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:43.709551 8960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:43.709564 8960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:43.709576 8960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:43.709589 8960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:43.710991 8960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:37911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:37911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:43.711194 8960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:43.711441 8960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:43.716315 8965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:43.716385 8966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:43.716538 8960 server_base.cc:1034] running on GCE node
W20260809 00:36:43.716333 8968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:43.716897 8960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:43.717187 8960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:43.718397 8960 hybrid_clock.cc:648] HybridClock initialized: now 1786235803718377 us; error 39 us; skew 500 ppm
I20260809 00:36:43.719671 8960 webserver.cc:460] Webserver started at http://127.21.41.254:44133/ using document root <none> and password file <none>
I20260809 00:36:43.719921 8960 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:43.719993 8960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:43.720171 8960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:43.721249 8960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "5073b932665b4f7b9e970d130aa85fa3"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "c09b76102f2f236aa614e5a055ca0f04"
server_key_iv: "67a4569089cddc33c9b2519806c53ba4"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.721561 8960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "5073b932665b4f7b9e970d130aa85fa3"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "c09b76102f2f236aa614e5a055ca0f04"
server_key_iv: "67a4569089cddc33c9b2519806c53ba4"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.722898 8960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:36:43.723572 8974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.723752 8960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.723843 8960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "5073b932665b4f7b9e970d130aa85fa3"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "c09b76102f2f236aa614e5a055ca0f04"
server_key_iv: "67a4569089cddc33c9b2519806c53ba4"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.723924 8960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:43.737344 8960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:43.737694 8960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:43.737857 8960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:43.742115 8960 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:37911
I20260809 00:36:43.742175 9026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:37911 every 8 connection(s)
I20260809 00:36:43.742555 8960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:43.743561 9027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:43.745862 9027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3: Bootstrap starting.
I20260809 00:36:43.746464 9027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:43.746717 9027 log.cc:824] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:43.747452 9027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3: No bootstrap required, opened a new log
I20260809 00:36:43.749023 9027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5073b932665b4f7b9e970d130aa85fa3" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37911 } }
I20260809 00:36:43.749161 9027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:43.749183 9027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5073b932665b4f7b9e970d130aa85fa3, State: Initialized, Role: FOLLOWER
I20260809 00:36:43.749284 9027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5073b932665b4f7b9e970d130aa85fa3" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37911 } }
I20260809 00:36:43.749369 9027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:43.749402 9027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:43.749459 9027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:43.749966 9027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5073b932665b4f7b9e970d130aa85fa3" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37911 } }
I20260809 00:36:43.750087 9027 leader_election.cc:302] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5073b932665b4f7b9e970d130aa85fa3; no voters:
I20260809 00:36:43.750276 9027 leader_election.cc:288] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:43.750330 9030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:43.750559 9027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:43.750591 9030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [term 1 LEADER]: Becoming Leader. State: Replica: 5073b932665b4f7b9e970d130aa85fa3, State: Running, Role: LEADER
I20260809 00:36:43.750867 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 8960
I20260809 00:36:43.750986 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:43.751194 21671 external_mini_cluster.cc:1428] Setting key eab15c3a050509408c3ecf8a7fe0252e
I20260809 00:36:43.751423 9030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5073b932665b4f7b9e970d130aa85fa3" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37911 } }
I20260809 00:36:43.752200 9032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5073b932665b4f7b9e970d130aa85fa3. Latest consensus state: current_term: 1 leader_uuid: "5073b932665b4f7b9e970d130aa85fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5073b932665b4f7b9e970d130aa85fa3" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37911 } } }
I20260809 00:36:43.752228 9031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5073b932665b4f7b9e970d130aa85fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5073b932665b4f7b9e970d130aa85fa3" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 37911 } } }
I20260809 00:36:43.752367 9032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:43.752373 9031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3 [sys.catalog]: This master's current role is: LEADER
W20260809 00:36:43.752532 9046 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:36:43.752580 9046 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:36:43.752678 9048 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:43.752842 9048 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:43.756075 9048 catalog_manager.cc:1347] Generated new cluster ID: 96b2753a542b4936aa46daa8ffd04247
I20260809 00:36:43.756127 9048 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:43.757287 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.751704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59982 (local address 127.21.41.254:37911)
0809 00:36:43.752464 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.752473 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.752506 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:36:43.752627 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.752636 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.752722 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.752919 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:43.753159 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.754146 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.755123 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.755770 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.755893 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.756600 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:43.756604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:43.756604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:43.756792 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:43.756824 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:43.756835 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:43.756923 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.756998 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.757078 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":86,"threads_started":1}
I20260809 00:36:43.767964 9048 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:43.768644 9048 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:43.775373 9048 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3: Generated new TSK 0
I20260809 00:36:43.775566 9048 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:43.784618 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.193:0
--local_ip_for_outbound_sockets=127.21.41.193
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:37911
--builtin_ntp_servers=127.21.41.212:42503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:36:43.869191 9051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:43.869416 9051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:43.869441 9051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:43.870816 9051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:43.870862 9051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:43.870891 9051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.193
I20260809 00:36:43.872267 9051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.41.193
--webserver_port=0
--tserver_master_addrs=127.21.41.254:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.193
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:43.872534 9051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:43.872849 9051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:43.878684 9056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:43.878719 9057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:43.878718 9059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:43.879418 9051 server_base.cc:1034] running on GCE node
I20260809 00:36:43.879643 9051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:43.879897 9051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:43.881067 9051 hybrid_clock.cc:648] HybridClock initialized: now 1786235803881029 us; error 53 us; skew 500 ppm
I20260809 00:36:43.882409 9051 webserver.cc:460] Webserver started at http://127.21.41.193:46101/ using document root <none> and password file <none>
I20260809 00:36:43.882669 9051 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:43.882742 9051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:43.882908 9051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:43.884011 9051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/data/instance:
uuid: "3dbb29c4e6724666ac07fe87533a0030"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "c11eeef2f7ce28c3bfd2e706ee8a1124"
server_key_iv: "8c15b4b2005dfb89b53e0a17672eb8bb"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.884387 9051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance:
uuid: "3dbb29c4e6724666ac07fe87533a0030"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "c11eeef2f7ce28c3bfd2e706ee8a1124"
server_key_iv: "8c15b4b2005dfb89b53e0a17672eb8bb"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.885727 9051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:36:43.886540 9065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.886716 9051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:43.886824 9051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/wal
uuid: "3dbb29c4e6724666ac07fe87533a0030"
format_stamp: "Formatted at 2026-08-09 00:36:43 on dist-test-slave-zpfg"
server_key: "c11eeef2f7ce28c3bfd2e706ee8a1124"
server_key_iv: "8c15b4b2005dfb89b53e0a17672eb8bb"
server_key_version: "encryptionkey@0"
I20260809 00:36:43.886916 9051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:43.901321 9051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:43.901654 9051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:43.901818 9051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:43.902062 9051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:43.902518 9051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:43.902575 9051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.902607 9051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:43.902634 9051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:43.908483 9051 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.193:36603
I20260809 00:36:43.908533 9178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.193:36603 every 8 connection(s)
I20260809 00:36:43.908879 9051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/data/info.pb
I20260809 00:36:43.910843 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 9051
I20260809 00:36:43.910944 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-0/wal/instance
I20260809 00:36:43.911155 21671 external_mini_cluster.cc:1428] Setting key eb34c4d8dde402e995f8cd2cc4a03b0e
I20260809 00:36:43.912225 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.194:0
--local_ip_for_outbound_sockets=127.21.41.194
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:37911
--builtin_ntp_servers=127.21.41.212:42503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 00:36:43.916196 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.909649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.193:44533 (local address 127.21.41.254:37911)
0809 00:36:43.909705 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:43.909707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:43.910083 (+ 376us) server_negotiation.cc:407] Connection header received
0809 00:36:43.910856 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:43.910858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:43.910877 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:43.911006 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:43.911976 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.913153 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.914547 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.914882 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.914965 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.915870 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:43.915871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:43.915872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:43.915877 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:43.915896 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:43.915901 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:43.915968 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:43.916042 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 00:36:43.916079 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:36:43.916556 9181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:43.909603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:37911 (local address 127.21.41.193:44533)
0809 00:36:43.909948 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:43.909992 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:43.910713 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:43.911134 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:43.911138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:43.911149 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:43.911864 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.911872 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.913271 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.913272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.914448 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:43.914451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:43.914976 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:43.914977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:43.915090 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:43.915717 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:36:43.915718 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:43.915749 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:36:43.915774 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:43.915986 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:43.915989 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:43.915991 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:36:43.916040 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 00:36:43.916122 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":114,"threads_started":1}
I20260809 00:36:43.917076 9179 heartbeater.cc:347] Connected to a master server at 127.21.41.254:37911
I20260809 00:36:43.917176 9179 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:43.917397 9179 heartbeater.cc:511] Master 127.21.41.254:37911 requested a full tablet report, sending...
I20260809 00:36:43.917932 8991 ts_manager.cc:194] Registered new tserver with Master: 3dbb29c4e6724666ac07fe87533a0030 (127.21.41.193:36603)
I20260809 00:36:43.918707 8991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.193:44533
W20260809 00:36:43.996980 9182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:43.997260 9182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:43.997303 9182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:43.998687 9182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:43.998757 9182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:43.998806 9182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.194
I20260809 00:36:44.000360 9182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.41.194
--webserver_port=0
--tserver_master_addrs=127.21.41.254:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.194
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:44.000665 9182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:44.001015 9182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:44.006397 9188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:44.006413 9190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:44.006534 9182 server_base.cc:1034] running on GCE node
W20260809 00:36:44.006407 9187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:44.006853 9182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:44.007128 9182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:44.008319 9182 hybrid_clock.cc:648] HybridClock initialized: now 1786235804008306 us; error 41 us; skew 500 ppm
I20260809 00:36:44.009766 9182 webserver.cc:460] Webserver started at http://127.21.41.194:36763/ using document root <none> and password file <none>
I20260809 00:36:44.010038 9182 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:44.010111 9182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:44.010268 9182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:44.011415 9182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/data/instance:
uuid: "4797f9db535242af8410a62a0f4ab51e"
format_stamp: "Formatted at 2026-08-09 00:36:44 on dist-test-slave-zpfg"
server_key: "c0c61c8ab5ef3e39ba2b4348079e68cd"
server_key_iv: "54c36da0ce42ba6d047540bbeff67ac9"
server_key_version: "encryptionkey@0"
I20260809 00:36:44.011790 9182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance:
uuid: "4797f9db535242af8410a62a0f4ab51e"
format_stamp: "Formatted at 2026-08-09 00:36:44 on dist-test-slave-zpfg"
server_key: "c0c61c8ab5ef3e39ba2b4348079e68cd"
server_key_iv: "54c36da0ce42ba6d047540bbeff67ac9"
server_key_version: "encryptionkey@0"
I20260809 00:36:44.013175 9182 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:36:44.014030 9196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:44.014273 9182 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:44.014393 9182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/wal
uuid: "4797f9db535242af8410a62a0f4ab51e"
format_stamp: "Formatted at 2026-08-09 00:36:44 on dist-test-slave-zpfg"
server_key: "c0c61c8ab5ef3e39ba2b4348079e68cd"
server_key_iv: "54c36da0ce42ba6d047540bbeff67ac9"
server_key_version: "encryptionkey@0"
I20260809 00:36:44.014484 9182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:44.024595 9182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:44.024993 9182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:44.025156 9182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:44.025394 9182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:44.025748 9182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:44.025837 9182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:44.025882 9182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:44.025909 9182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:44.032203 9182 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.194:33601
I20260809 00:36:44.032258 9309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.194:33601 every 8 connection(s)
I20260809 00:36:44.032577 9182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/data/info.pb
I20260809 00:36:44.038166 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 9182
I20260809 00:36:44.038251 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/wal/instance
I20260809 00:36:44.038501 21671 external_mini_cluster.cc:1428] Setting key eaec36a09fc51413900169622db442e7
I20260809 00:36:44.038753 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.033344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.194:45967 (local address 127.21.41.254:37911)
0809 00:36:44.033453 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:44.033456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:44.033750 (+ 294us) server_negotiation.cc:407] Connection header received
0809 00:36:44.034484 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:44.034485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:44.034499 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:44.034571 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:44.035353 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.035940 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.037054 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.037296 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.037385 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.038443 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:44.038445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:44.038445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:44.038450 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:44.038467 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:44.038471 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:44.038546 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:44.038643 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 00:36:44.038680 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 00:36:44.039014 9312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.033297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:37911 (local address 127.21.41.194:45967)
0809 00:36:44.033538 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:44.033577 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:44.034315 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:44.034650 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:44.034655 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:44.034672 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:44.035240 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.035245 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.036054 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.036056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.036944 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.036946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.037395 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.037396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.037546 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.038209 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:36:44.038209 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:44.038274 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:36:44.038305 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:44.038550 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:44.038556 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:44.038559 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:36:44.038636 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:36:44.038754 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260809 00:36:44.039561 9310 heartbeater.cc:347] Connected to a master server at 127.21.41.254:37911
I20260809 00:36:44.039664 9310 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:44.039673 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.41.195:0
--local_ip_for_outbound_sockets=127.21.41.195
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:37911
--builtin_ntp_servers=127.21.41.212:42503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 00:36:44.039901 9310 heartbeater.cc:511] Master 127.21.41.254:37911 requested a full tablet report, sending...
I20260809 00:36:44.040284 8991 ts_manager.cc:194] Registered new tserver with Master: 4797f9db535242af8410a62a0f4ab51e (127.21.41.194:33601)
I20260809 00:36:44.040791 8991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.194:45967
W20260809 00:36:44.123101 9313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:44.123322 9313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:44.123343 9313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:44.124439 9313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:44.124472 9313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:44.124497 9313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.41.195
I20260809 00:36:44.125809 9313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:42503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.41.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.41.195
--webserver_port=0
--tserver_master_addrs=127.21.41.254:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.41.195
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:44.125998 9313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:44.126374 9313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:44.131474 9318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:44.131460 9321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:44.131453 9319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:44.131742 9313 server_base.cc:1034] running on GCE node
I20260809 00:36:44.131949 9313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:44.132143 9313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:44.133282 9313 hybrid_clock.cc:648] HybridClock initialized: now 1786235804133268 us; error 29 us; skew 500 ppm
I20260809 00:36:44.134835 9313 webserver.cc:460] Webserver started at http://127.21.41.195:44775/ using document root <none> and password file <none>
I20260809 00:36:44.135092 9313 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:44.135138 9313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:44.135273 9313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:44.136345 9313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/data/instance:
uuid: "3bae4186e59344da978209767ec11c44"
format_stamp: "Formatted at 2026-08-09 00:36:44 on dist-test-slave-zpfg"
server_key: "4e55867b6b9704a69ff9704b3d3054e9"
server_key_iv: "640a0ec74ada4d6bc052d0857edca844"
server_key_version: "encryptionkey@0"
I20260809 00:36:44.136698 9313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance:
uuid: "3bae4186e59344da978209767ec11c44"
format_stamp: "Formatted at 2026-08-09 00:36:44 on dist-test-slave-zpfg"
server_key: "4e55867b6b9704a69ff9704b3d3054e9"
server_key_iv: "640a0ec74ada4d6bc052d0857edca844"
server_key_version: "encryptionkey@0"
I20260809 00:36:44.137998 9313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:36:44.138772 9327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:44.138936 9313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:44.139030 9313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/wal
uuid: "3bae4186e59344da978209767ec11c44"
format_stamp: "Formatted at 2026-08-09 00:36:44 on dist-test-slave-zpfg"
server_key: "4e55867b6b9704a69ff9704b3d3054e9"
server_key_iv: "640a0ec74ada4d6bc052d0857edca844"
server_key_version: "encryptionkey@0"
I20260809 00:36:44.139086 9313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:44.164692 9313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:44.164981 9313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:44.165088 9313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:44.165370 9313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:36:44.165750 9313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:36:44.165786 9313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:44.165808 9313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:36:44.165858 9313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:44.171641 9313 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.195:39455
I20260809 00:36:44.171691 9440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.195:39455 every 8 connection(s)
I20260809 00:36:44.172029 9313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/data/info.pb
I20260809 00:36:44.175442 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 9313
I20260809 00:36:44.175531 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/wal/instance
I20260809 00:36:44.175770 21671 external_mini_cluster.cc:1428] Setting key 647fac5141bd2e8cb5d35a61171a7ec3
I20260809 00:36:44.177856 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.172812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:58503 (local address 127.21.41.254:37911)
0809 00:36:44.172887 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:44.172889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:44.173194 (+ 305us) server_negotiation.cc:407] Connection header received
0809 00:36:44.173784 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:44.173785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:44.173797 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:44.173867 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:44.174587 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.175080 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.176310 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.176516 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.176602 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.177566 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:44.177567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:44.177567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:44.177571 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:44.177585 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:44.177589 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:44.177699 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:44.177769 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 00:36:44.177798 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:36:44.178061 9443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.172806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:37911 (local address 127.21.41.195:58503)
0809 00:36:44.173028 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:44.173091 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:44.173598 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:44.173952 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:44.173956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:44.173965 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:36:44.174500 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.174503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.175159 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.175159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.176172 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.176174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.176711 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.176713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.176909 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.177438 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:36:44.177439 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:36:44.177467 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:36:44.177489 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:36:44.177704 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:36:44.177708 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:36:44.177710 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:36:44.177762 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:36:44.177841 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":123,"threads_started":1}
I20260809 00:36:44.178666 9441 heartbeater.cc:347] Connected to a master server at 127.21.41.254:37911
I20260809 00:36:44.178754 9441 heartbeater.cc:464] Registering TS with master...
I20260809 00:36:44.178903 9441 heartbeater.cc:511] Master 127.21.41.254:37911 requested a full tablet report, sending...
I20260809 00:36:44.179288 8991 ts_manager.cc:194] Registered new tserver with Master: 3bae4186e59344da978209767ec11c44 (127.21.41.195:39455)
I20260809 00:36:44.179610 8991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.41.195:58503
I20260809 00:36:44.187968 21671 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:36:44.192631 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.189443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59984 (local address 127.21.41.254:37911)
0809 00:36:44.189509 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:44.189511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:44.189697 (+ 186us) server_negotiation.cc:407] Connection header received
0809 00:36:44.189723 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:44.189724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:44.189739 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:44.189807 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:44.189980 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.190590 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.191367 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.191579 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.191652 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.192433 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:44.192434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:44.192434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:44.192438 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:44.192452 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:44.192456 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:44.192505 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:44.192541 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 00:36:44.192565 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:36:44.194633 8991 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:36:44.196481 8991 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:36:44.203231 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.199359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:36603 (local address 127.0.0.1:56964)
0809 00:36:44.199531 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:44.199541 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:44.199599 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:44.199896 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:44.199897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:44.199901 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:44.199948 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.199951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.200807 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.200808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.201829 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.201841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.202896 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.202898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.202981 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.202996 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:36:44.203080 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:36:44.203143 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":81,"threads_started":1}
I20260809 00:36:44.203245 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.199451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56964 (local address 127.21.41.193:36603)
0809 00:36:44.199649 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:44.199655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:44.199665 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:36:44.199706 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:44.199708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:44.199724 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:44.199811 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:44.200010 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.200717 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.201934 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.202811 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.202886 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.202992 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:44.203095 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 00:36:44.203140 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
I20260809 00:36:44.203449 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.199126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.195:39455 (local address 127.0.0.1:46078)
0809 00:36:44.199277 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:44.199290 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:44.199349 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:44.199924 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:44.199925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:44.199927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:44.199969 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.199971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.201385 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.201386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.202277 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.202279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.203195 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.203196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.203259 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.203263 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:36:44.203293 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 00:36:44.203387 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":30,"thread_start_us":90,"threads_started":1}
I20260809 00:36:44.203845 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.199572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46078 (local address 127.21.41.195:39455)
0809 00:36:44.199769 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:44.199773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:44.199782 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:36:44.199817 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:44.199819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:44.199832 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:44.199909 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:44.200072 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.201281 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.202407 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.203105 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.203595 (+ 490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.203691 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:44.203715 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 00:36:44.203755 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":81,"threads_started":1}
I20260809 00:36:44.204059 9113 tablet_service.cc:1467] Processing CreateTablet for tablet 52e14de909fb406b8cabff4d749f0de8 (DEFAULT_TABLE table=test-table [id=2d772fdde9e14b6cb9dd1e9eedbf44ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:44.204345 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.198894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:33601 (local address 127.0.0.1:54562)
0809 00:36:44.199024 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:44.199071 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:44.199135 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:44.199781 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:44.199784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:44.199787 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:44.199863 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.199864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.201410 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.201411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.202709 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.202712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.204070 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.204072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.204141 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.204155 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:36:44.204219 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 00:36:44.204264 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":83,"threads_started":1}
I20260809 00:36:44.204366 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.198991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54562 (local address 127.21.41.194:33601)
0809 00:36:44.199229 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:44.199236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:44.199250 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:36:44.199299 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:44.199303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:44.199322 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:44.199411 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:44.199981 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.201305 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.202820 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.203851 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.203925 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.204025 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:44.204247 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 00:36:44.204286 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":117,"threads_started":1}
I20260809 00:36:44.204525 9113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52e14de909fb406b8cabff4d749f0de8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:44.204499 9375 tablet_service.cc:1467] Processing CreateTablet for tablet 52e14de909fb406b8cabff4d749f0de8 (DEFAULT_TABLE table=test-table [id=2d772fdde9e14b6cb9dd1e9eedbf44ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:44.204795 9375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52e14de909fb406b8cabff4d749f0de8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:44.205116 9244 tablet_service.cc:1467] Processing CreateTablet for tablet 52e14de909fb406b8cabff4d749f0de8 (DEFAULT_TABLE table=test-table [id=2d772fdde9e14b6cb9dd1e9eedbf44ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:36:44.205425 9244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52e14de909fb406b8cabff4d749f0de8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:44.208024 9455 tablet_bootstrap.cc:492] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030: Bootstrap starting.
I20260809 00:36:44.208139 9456 tablet_bootstrap.cc:492] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44: Bootstrap starting.
I20260809 00:36:44.208612 9457 tablet_bootstrap.cc:492] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: Bootstrap starting.
I20260809 00:36:44.208889 9455 tablet_bootstrap.cc:654] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:44.208889 9456 tablet_bootstrap.cc:654] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:44.209266 9457 tablet_bootstrap.cc:654] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:44.209280 9455 log.cc:824] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:44.209280 9456 log.cc:824] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:44.209625 9457 log.cc:824] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:44.210145 9455 tablet_bootstrap.cc:492] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030: No bootstrap required, opened a new log
I20260809 00:36:44.210161 9456 tablet_bootstrap.cc:492] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44: No bootstrap required, opened a new log
I20260809 00:36:44.210225 9455 ts_tablet_manager.cc:1397] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:36:44.210227 9456 ts_tablet_manager.cc:1397] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:36:44.210402 9457 tablet_bootstrap.cc:492] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: No bootstrap required, opened a new log
I20260809 00:36:44.210472 9457 ts_tablet_manager.cc:1397] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:36:44.212270 9455 raft_consensus.cc:348] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4797f9db535242af8410a62a0f4ab51e" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 33601 } } peers { permanent_uuid: "3bae4186e59344da978209767ec11c44" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 39455 } } peers { permanent_uuid: "3dbb29c4e6724666ac07fe87533a0030" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36603 } }
I20260809 00:36:44.212423 9455 raft_consensus.cc:374] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:44.212467 9455 raft_consensus.cc:729] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dbb29c4e6724666ac07fe87533a0030, State: Initialized, Role: FOLLOWER
I20260809 00:36:44.212601 9455 consensus_queue.cc:260] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4797f9db535242af8410a62a0f4ab51e" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 33601 } } peers { permanent_uuid: "3bae4186e59344da978209767ec11c44" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 39455 } } peers { permanent_uuid: "3dbb29c4e6724666ac07fe87533a0030" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36603 } }
I20260809 00:36:44.212875 9455 ts_tablet_manager.cc:1428] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:36:44.213071 9179 heartbeater.cc:503] Master 127.21.41.254:37911 was elected leader, sending a full tablet report...
I20260809 00:36:44.213217 9456 raft_consensus.cc:348] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4797f9db535242af8410a62a0f4ab51e" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 33601 } } peers { permanent_uuid: "3bae4186e59344da978209767ec11c44" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 39455 } } peers { permanent_uuid: "3dbb29c4e6724666ac07fe87533a0030" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36603 } }
I20260809 00:36:44.213419 9456 raft_consensus.cc:374] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:44.213469 9456 raft_consensus.cc:729] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bae4186e59344da978209767ec11c44, State: Initialized, Role: FOLLOWER
I20260809 00:36:44.213405 9457 raft_consensus.cc:348] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4797f9db535242af8410a62a0f4ab51e" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 33601 } } peers { permanent_uuid: "3bae4186e59344da978209767ec11c44" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 39455 } } peers { permanent_uuid: "3dbb29c4e6724666ac07fe87533a0030" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36603 } }
I20260809 00:36:44.213565 9457 raft_consensus.cc:374] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:44.213613 9457 raft_consensus.cc:729] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4797f9db535242af8410a62a0f4ab51e, State: Initialized, Role: FOLLOWER
I20260809 00:36:44.213640 9456 consensus_queue.cc:260] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4797f9db535242af8410a62a0f4ab51e" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 33601 } } peers { permanent_uuid: "3bae4186e59344da978209767ec11c44" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 39455 } } peers { permanent_uuid: "3dbb29c4e6724666ac07fe87533a0030" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36603 } }
I20260809 00:36:44.213725 9457 consensus_queue.cc:260] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4797f9db535242af8410a62a0f4ab51e" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 33601 } } peers { permanent_uuid: "3bae4186e59344da978209767ec11c44" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 39455 } } peers { permanent_uuid: "3dbb29c4e6724666ac07fe87533a0030" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36603 } }
I20260809 00:36:44.213939 9456 ts_tablet_manager.cc:1428] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 00:36:44.213987 9457 ts_tablet_manager.cc:1428] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 00:36:44.214080 9441 heartbeater.cc:503] Master 127.21.41.254:37911 was elected leader, sending a full tablet report...
I20260809 00:36:44.214383 9310 heartbeater.cc:503] Master 127.21.41.254:37911 was elected leader, sending a full tablet report...
I20260809 00:36:44.278260 9462 raft_consensus.cc:482] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:36:44.278469 9462 raft_consensus.cc:504] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4797f9db535242af8410a62a0f4ab51e" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 33601 } } peers { permanent_uuid: "3bae4186e59344da978209767ec11c44" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 39455 } } peers { permanent_uuid: "3dbb29c4e6724666ac07fe87533a0030" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36603 } }
I20260809 00:36:44.278833 9462 leader_election.cc:288] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4797f9db535242af8410a62a0f4ab51e (127.21.41.194:33601), 3dbb29c4e6724666ac07fe87533a0030 (127.21.41.193:36603)
I20260809 00:36:44.281886 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.279122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:37223 (local address 127.21.41.194:33601)
0809 00:36:44.279178 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:44.279180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:44.279192 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:36:44.279226 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:44.279227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:44.279239 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:44.279338 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:44.279555 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.280125 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.281166 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.281522 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.281578 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.281584 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:44.281762 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:36:44.281811 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:36:44.281908 9443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.279031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.194:33601 (local address 127.21.41.195:37223)
0809 00:36:44.279092 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:44.279106 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:44.279152 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:44.279365 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:44.279367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:44.279368 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:44.279466 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.279468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.280261 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.280262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.281070 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.281072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.281616 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.281617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.281686 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.281689 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:36:44.281747 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:36:44.281778 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":53}
I20260809 00:36:44.281937 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.279272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.41.195:43795 (local address 127.21.41.193:36603)
0809 00:36:44.279317 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:44.279319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:44.279421 (+ 102us) server_negotiation.cc:407] Connection header received
0809 00:36:44.279461 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:44.279462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:44.279474 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:44.279532 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:36:44.279670 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.280123 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.281168 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.281536 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.281653 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.281661 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:44.281825 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 00:36:44.281869 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:36:44.282038 9464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.279159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:36603 (local address 127.21.41.195:43795)
0809 00:36:44.279343 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:44.279355 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:36:44.279414 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:36:44.279548 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:36:44.279549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:36:44.279550 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:36:44.279616 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.279617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.280220 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.280221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.281070 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:36:44.281072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.281674 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.281675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:36:44.281743 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.281748 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:36:44.281806 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:36:44.281838 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":29,"thread_start_us":94,"threads_started":1}
I20260809 00:36:44.282378 9133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52e14de909fb406b8cabff4d749f0de8" candidate_uuid: "3bae4186e59344da978209767ec11c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dbb29c4e6724666ac07fe87533a0030" is_pre_election: true
I20260809 00:36:44.282387 9264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52e14de909fb406b8cabff4d749f0de8" candidate_uuid: "3bae4186e59344da978209767ec11c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4797f9db535242af8410a62a0f4ab51e" is_pre_election: true
I20260809 00:36:44.282541 9133 raft_consensus.cc:2454] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bae4186e59344da978209767ec11c44 in term 0.
I20260809 00:36:44.282547 9264 raft_consensus.cc:2454] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bae4186e59344da978209767ec11c44 in term 0.
I20260809 00:36:44.282755 9328 leader_election.cc:302] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bae4186e59344da978209767ec11c44, 4797f9db535242af8410a62a0f4ab51e; no voters:
I20260809 00:36:44.282878 9462 raft_consensus.cc:2781] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:36:44.282932 9462 raft_consensus.cc:482] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:36:44.282959 9462 raft_consensus.cc:3037] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 0 FOLLOWER]: Advancing to term 1
W20260809 00:36:44.283118 9311 tablet.cc:2369] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:36:44.283736 9462 raft_consensus.cc:504] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4797f9db535242af8410a62a0f4ab51e" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 33601 } } peers { permanent_uuid: "3bae4186e59344da978209767ec11c44" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 39455 } } peers { permanent_uuid: "3dbb29c4e6724666ac07fe87533a0030" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36603 } }
I20260809 00:36:44.283875 9462 leader_election.cc:288] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [CANDIDATE]: Term 1 election: Requested vote from peers 4797f9db535242af8410a62a0f4ab51e (127.21.41.194:33601), 3dbb29c4e6724666ac07fe87533a0030 (127.21.41.193:36603)
I20260809 00:36:44.284015 9264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52e14de909fb406b8cabff4d749f0de8" candidate_uuid: "3bae4186e59344da978209767ec11c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4797f9db535242af8410a62a0f4ab51e"
I20260809 00:36:44.284057 9133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52e14de909fb406b8cabff4d749f0de8" candidate_uuid: "3bae4186e59344da978209767ec11c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dbb29c4e6724666ac07fe87533a0030"
I20260809 00:36:44.284107 9264 raft_consensus.cc:3037] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:44.284150 9133 raft_consensus.cc:3037] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:44.284926 9264 raft_consensus.cc:2454] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bae4186e59344da978209767ec11c44 in term 1.
I20260809 00:36:44.284988 9133 raft_consensus.cc:2454] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bae4186e59344da978209767ec11c44 in term 1.
I20260809 00:36:44.285120 9328 leader_election.cc:302] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bae4186e59344da978209767ec11c44, 4797f9db535242af8410a62a0f4ab51e; no voters:
I20260809 00:36:44.285202 9462 raft_consensus.cc:2781] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:44.285349 9462 raft_consensus.cc:686] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 1 LEADER]: Becoming Leader. State: Replica: 3bae4186e59344da978209767ec11c44, State: Running, Role: LEADER
I20260809 00:36:44.285444 9462 consensus_queue.cc:237] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4797f9db535242af8410a62a0f4ab51e" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 33601 } } peers { permanent_uuid: "3bae4186e59344da978209767ec11c44" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 39455 } } peers { permanent_uuid: "3dbb29c4e6724666ac07fe87533a0030" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36603 } }
I20260809 00:36:44.286405 8991 catalog_manager.cc:5495] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bae4186e59344da978209767ec11c44 (127.21.41.195). New cstate: current_term: 1 leader_uuid: "3bae4186e59344da978209767ec11c44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4797f9db535242af8410a62a0f4ab51e" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 33601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bae4186e59344da978209767ec11c44" member_type: VOTER last_known_addr { host: "127.21.41.195" port: 39455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3dbb29c4e6724666ac07fe87533a0030" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36603 } health_report { overall_health: UNKNOWN } } }
I20260809 00:36:44.295796 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.293578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46086 (local address 127.21.41.195:39455)
0809 00:36:44.293640 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:44.293641 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:44.293648 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:36:44.293677 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:44.293678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:44.293692 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:44.293751 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:36:44.293919 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.294408 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.295129 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:44.295302 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:44.295349 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:44.295501 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:36:44.295594 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:36:44.295664 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:44.295709 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 00:36:44.295723 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:36:44.297650 9133 raft_consensus.cc:1261] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030 [term 1 FOLLOWER]: Refusing update from remote peer 3bae4186e59344da978209767ec11c44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:44.297667 9264 raft_consensus.cc:1261] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e [term 1 FOLLOWER]: Refusing update from remote peer 3bae4186e59344da978209767ec11c44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:36:44.298046 9465 consensus_queue.cc:1035] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dbb29c4e6724666ac07fe87533a0030" member_type: VOTER last_known_addr { host: "127.21.41.193" port: 36603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:44.298179 9462 consensus_queue.cc:1035] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4797f9db535242af8410a62a0f4ab51e" member_type: VOTER last_known_addr { host: "127.21.41.194" port: 33601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:36:44.300554 9468 mvcc.cc:205] Tried to move back new op lower bound from 7316421854400364544 to 7316421854353928192. Current Snapshot: MvccSnapshot[applied={T|T < 7316421854400364544}]
I20260809 00:36:44.301193 9469 mvcc.cc:205] Tried to move back new op lower bound from 7316421854400364544 to 7316421854353928192. Current Snapshot: MvccSnapshot[applied={T|T < 7316421854400364544}]
I20260809 00:36:44.303102 8991 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:44.303202 8991 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:36:44.303998 8991 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5073b932665b4f7b9e970d130aa85fa3: Sending DeleteTablet for 3 replicas of tablet 52e14de909fb406b8cabff4d749f0de8
I20260809 00:36:44.304316 9244 tablet_service.cc:1514] Processing DeleteTablet for tablet 52e14de909fb406b8cabff4d749f0de8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:44 UTC) from {username='slave'} at 127.0.0.1:54562
I20260809 00:36:44.304414 9113 tablet_service.cc:1514] Processing DeleteTablet for tablet 52e14de909fb406b8cabff4d749f0de8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:44 UTC) from {username='slave'} at 127.0.0.1:56964
I20260809 00:36:44.304553 9375 tablet_service.cc:1514] Processing DeleteTablet for tablet 52e14de909fb406b8cabff4d749f0de8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:36:44 UTC) from {username='slave'} at 127.0.0.1:46078
I20260809 00:36:44.304605 9478 tablet_replica.cc:330] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: stopping tablet replica
I20260809 00:36:44.304708 9478 raft_consensus.cc:2229] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:44.304809 9480 tablet_replica.cc:330] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44: stopping tablet replica
I20260809 00:36:44.304819 9478 raft_consensus.cc:2258] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:44.304965 9479 tablet_replica.cc:330] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030: stopping tablet replica
I20260809 00:36:44.305035 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 9051
I20260809 00:36:44.305078 9479 raft_consensus.cc:2229] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:36:44.305158 9480 raft_consensus.cc:2229] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:36:44.305187 9479 raft_consensus.cc:2258] T 52e14de909fb406b8cabff4d749f0de8 P 3dbb29c4e6724666ac07fe87533a0030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:44.305243 9478 ts_tablet_manager.cc:1905] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:36:44.305780 9480 raft_consensus.cc:2258] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:36:44.306289 9480 ts_tablet_manager.cc:1905] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:36:44.307264 9478 ts_tablet_manager.cc:1918] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:44.307353 9478 log.cc:1196] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-1/wal/wals/52e14de909fb406b8cabff4d749f0de8
I20260809 00:36:44.307610 9478 ts_tablet_manager.cc:1939] T 52e14de909fb406b8cabff4d749f0de8 P 4797f9db535242af8410a62a0f4ab51e: Deleting consensus metadata
I20260809 00:36:44.308034 8975 catalog_manager.cc:4841] TS 4797f9db535242af8410a62a0f4ab51e (127.21.41.194:33601): tablet 52e14de909fb406b8cabff4d749f0de8 (table test-table [id=2d772fdde9e14b6cb9dd1e9eedbf44ee]) successfully deleted
I20260809 00:36:44.308228 9480 ts_tablet_manager.cc:1918] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:36:44.308311 9480 log.cc:1196] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44: Deleting WAL directory at /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786235559853195-21671-0/minicluster-data/ts-2/wal/wals/52e14de909fb406b8cabff4d749f0de8
I20260809 00:36:44.308542 9480 ts_tablet_manager.cc:1939] T 52e14de909fb406b8cabff4d749f0de8 P 3bae4186e59344da978209767ec11c44: Deleting consensus metadata
I20260809 00:36:44.308936 8975 catalog_manager.cc:4841] TS 3bae4186e59344da978209767ec11c44 (127.21.41.195:39455): tablet 52e14de909fb406b8cabff4d749f0de8 (table test-table [id=2d772fdde9e14b6cb9dd1e9eedbf44ee]) successfully deleted
W20260809 00:36:44.311847 8975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.41.193:36603 (error 108)
I20260809 00:36:44.312065 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 9182
I20260809 00:36:44.312719 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:44.312221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.193:36603 (local address 127.0.0.1:56972)
0809 00:36:44.312625 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 00:36:44.312679 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.193:36603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381}
W20260809 00:36:44.313146 8975 catalog_manager.cc:4584] TS 3dbb29c4e6724666ac07fe87533a0030 (127.21.41.193:36603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52e14de909fb406b8cabff4d749f0de8: Network error: Client connection negotiation failed: client connection to 127.21.41.193:36603: connect: Connection refused (error 111)
I20260809 00:36:44.317188 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 9313
I20260809 00:36:44.321573 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 8960
2026-08-09T00:36:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (722 ms)
[----------] 6 tests from EncryptionPolicyTest (8703 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:44 dist-test-slave-zpfg krb5kdc[9484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:44 dist-test-slave-zpfg krb5kdc[9484](info): set up 2 sockets
Aug 09 00:36:44 dist-test-slave-zpfg krb5kdc[9484](info): commencing operation
krb5kdc: starting...
W20260809 00:36:45.480026 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:45 dist-test-slave-zpfg krb5kdc[9484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:36:45.605247 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:45.689383 9497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:45.689638 9497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:45.689687 9497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:45.690896 9497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:45.690966 9497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:45.690990 9497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:45.691012 9497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:45.691042 9497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:45.692514 9497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44187
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:45.692816 9497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:45.693049 9497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:45.698005 9503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:45.698081 9502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:45.698191 9497 server_base.cc:1034] running on GCE node
W20260809 00:36:45.698243 9505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:45.698773 9497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:45.699033 9497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:45.700212 9497 hybrid_clock.cc:648] HybridClock initialized: now 1786235805700200 us; error 30 us; skew 500 ppm
Aug 09 00:36:45 dist-test-slave-zpfg krb5kdc[9484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235805, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:45.702734 9497 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:36:45.703212 9497 webserver.cc:460] Webserver started at http://127.0.0.1:37691/ using document root <none> and password file <none>
I20260809 00:36:45.703457 9497 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:45.703504 9497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:45.703648 9497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:45.704759 9497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "783f553caaf14846be239a95aabb50fe"
format_stamp: "Formatted at 2026-08-09 00:36:45 on dist-test-slave-zpfg"
server_key: "f6c7143938cd925464dfbfe4a44e1a2c"
server_key_iv: "d2938b22a6fcac53cfc03cf4af23cd70"
server_key_version: "encryptionkey@0"
I20260809 00:36:45.705137 9497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "783f553caaf14846be239a95aabb50fe"
format_stamp: "Formatted at 2026-08-09 00:36:45 on dist-test-slave-zpfg"
server_key: "f6c7143938cd925464dfbfe4a44e1a2c"
server_key_iv: "d2938b22a6fcac53cfc03cf4af23cd70"
server_key_version: "encryptionkey@0"
I20260809 00:36:45.706642 9497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:36:45.707460 9512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:45.707653 9497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:45.707763 9497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "783f553caaf14846be239a95aabb50fe"
format_stamp: "Formatted at 2026-08-09 00:36:45 on dist-test-slave-zpfg"
server_key: "f6c7143938cd925464dfbfe4a44e1a2c"
server_key_iv: "d2938b22a6fcac53cfc03cf4af23cd70"
server_key_version: "encryptionkey@0"
I20260809 00:36:45.707837 9497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:45.730110 9497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:45.732208 9497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:45.732407 9497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:45.736601 9497 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44187
I20260809 00:36:45.736641 9564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44187 every 8 connection(s)
I20260809 00:36:45.737035 9497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:45.738046 9565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:45.740404 9565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe: Bootstrap starting.
I20260809 00:36:45.740664 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 9497
I20260809 00:36:45.740797 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:45.741010 21671 external_mini_cluster.cc:1428] Setting key dced3e1312e7b87e4ef595ce8e643006
I20260809 00:36:45.741158 9565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:45.741500 9565 log.cc:824] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:45.742295 9565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe: No bootstrap required, opened a new log
I20260809 00:36:45.744186 9565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783f553caaf14846be239a95aabb50fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } }
I20260809 00:36:45.744359 9565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:45.744408 9565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783f553caaf14846be239a95aabb50fe, State: Initialized, Role: FOLLOWER
I20260809 00:36:45.744534 9565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783f553caaf14846be239a95aabb50fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } }
I20260809 00:36:45.744637 9565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:45.744691 9565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:45.744745 9565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:45.745327 9565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783f553caaf14846be239a95aabb50fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } }
I20260809 00:36:45.745462 9565 leader_election.cc:302] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 783f553caaf14846be239a95aabb50fe; no voters:
I20260809 00:36:45.745666 9565 leader_election.cc:288] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:45.745714 9570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:45.745857 9570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [term 1 LEADER]: Becoming Leader. State: Replica: 783f553caaf14846be239a95aabb50fe, State: Running, Role: LEADER
I20260809 00:36:45.745942 9565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:45.745950 9570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783f553caaf14846be239a95aabb50fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } }
I20260809 00:36:45.746376 9570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 783f553caaf14846be239a95aabb50fe. Latest consensus state: current_term: 1 leader_uuid: "783f553caaf14846be239a95aabb50fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783f553caaf14846be239a95aabb50fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } } }
I20260809 00:36:45.746454 9570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:45.747397 9571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "783f553caaf14846be239a95aabb50fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783f553caaf14846be239a95aabb50fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } } }
I20260809 00:36:45.747545 9571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:45.747586 9579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:45.748111 9579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 00:36:45 dist-test-slave-zpfg krb5kdc[9484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 00:36:45.750381 9579 catalog_manager.cc:1347] Generated new cluster ID: d6944467db674ddb94616c5bc5bc1c97
I20260809 00:36:45.750432 9579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:45.752890 9568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:45.741591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36871 (local address 127.0.0.1:44187)
0809 00:36:45.741810 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:45.741819 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:45.741853 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:36:45.742404 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:45.742414 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:45.742486 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:45.742634 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:45.743405 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:45.744320 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:45.745310 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:45.745945 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:45.746667 (+ 722us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:45.749110 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:45.749114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:45.749114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:45.749127 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:45.751174 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:45.751654 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:45.751657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:45.751657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:45.751757 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:45.752004 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:45.752005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:45.752005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:45.752288 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:45.752440 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:45.752585 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 00:36:45.752679 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":95,"threads_started":1}
I20260809 00:36:45.753973 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:36:45.770020 9568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:45.762675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58927 (local address 127.0.0.1:44187)
0809 00:36:45.762718 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:45.762720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:45.762822 (+ 102us) server_negotiation.cc:407] Connection header received
0809 00:36:45.762858 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:45.762859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:45.762870 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:45.762939 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:45.763559 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:45.764343 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:45.765363 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:45.765697 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:45.765772 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:45.767159 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:45.767160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:45.767161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:45.767168 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:45.768776 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:45.769203 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:45.769204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:45.769204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:45.769267 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:45.769490 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:45.769491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:45.769492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:45.769625 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:45.769737 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:45.769854 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 00:36:45.769922 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:36:45.771575 9579 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:45.772157 9579 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:45.776319 9579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 783f553caaf14846be239a95aabb50fe: Generated new TSK 0
I20260809 00:36:45.776510 9579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:45.780613 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 9497
W20260809 00:36:45.785427 9482 connection.cc:535] client connection to 127.0.0.1:44187 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44187): Connection reset by peer (error 104)
2026-08-09T00:36:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1457 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:45 dist-test-slave-zpfg krb5kdc[9596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:45 dist-test-slave-zpfg krb5kdc[9596](info): set up 2 sockets
Aug 09 00:36:45 dist-test-slave-zpfg krb5kdc[9596](info): commencing operation
krb5kdc: starting...
W20260809 00:36:46.827983 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:46 dist-test-slave-zpfg krb5kdc[9596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:36:46.953009 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:47.045531 9609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:47.045754 9609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:47.045783 9609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:47.047029 9609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:47.047068 9609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:47.047084 9609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:47.047098 9609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:47.047112 9609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:47.048830 9609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33115
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:47.049037 9609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:47.049239 9609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:47.054620 9615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:47.054629 9614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:47.054765 9609 server_base.cc:1034] running on GCE node
W20260809 00:36:47.054979 9617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:47.055240 9609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:47.055533 9609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:47.056732 9609 hybrid_clock.cc:648] HybridClock initialized: now 1786235807056716 us; error 34 us; skew 500 ppm
Aug 09 00:36:47 dist-test-slave-zpfg krb5kdc[9596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235807, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:47.059336 9609 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:36:47.059857 9609 webserver.cc:460] Webserver started at http://127.0.0.1:34465/ using document root <none> and password file <none>
I20260809 00:36:47.060111 9609 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:47.060180 9609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:47.060328 9609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:47.061422 9609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "2f98dee122a6451e92ca7605195767a4"
format_stamp: "Formatted at 2026-08-09 00:36:47 on dist-test-slave-zpfg"
server_key: "c0f5eca479d41b9f5cbe86ebd1d0e161"
server_key_iv: "56acd0a21d4605b457b86b215174d139"
server_key_version: "encryptionkey@0"
I20260809 00:36:47.061802 9609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "2f98dee122a6451e92ca7605195767a4"
format_stamp: "Formatted at 2026-08-09 00:36:47 on dist-test-slave-zpfg"
server_key: "c0f5eca479d41b9f5cbe86ebd1d0e161"
server_key_iv: "56acd0a21d4605b457b86b215174d139"
server_key_version: "encryptionkey@0"
I20260809 00:36:47.063314 9609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:36:47.064141 9624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:47.064370 9609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:36:47.064481 9609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "2f98dee122a6451e92ca7605195767a4"
format_stamp: "Formatted at 2026-08-09 00:36:47 on dist-test-slave-zpfg"
server_key: "c0f5eca479d41b9f5cbe86ebd1d0e161"
server_key_iv: "56acd0a21d4605b457b86b215174d139"
server_key_version: "encryptionkey@0"
I20260809 00:36:47.064558 9609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:47.073086 9609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:47.075067 9609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:47.075235 9609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:47.079751 9609 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33115
I20260809 00:36:47.079790 9676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33115 every 8 connection(s)
I20260809 00:36:47.080148 9609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:47.081013 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:47.083388 9677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4: Bootstrap starting.
I20260809 00:36:47.083997 9677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:47.084296 9677 log.cc:824] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:47.085002 9677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4: No bootstrap required, opened a new log
I20260809 00:36:47.086630 9677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f98dee122a6451e92ca7605195767a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260809 00:36:47.086778 9677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:47.086838 9677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f98dee122a6451e92ca7605195767a4, State: Initialized, Role: FOLLOWER
I20260809 00:36:47.086990 9677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f98dee122a6451e92ca7605195767a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260809 00:36:47.087054 9677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:47.087109 9677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:47.087162 9677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:47.087720 9677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f98dee122a6451e92ca7605195767a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260809 00:36:47.087842 9677 leader_election.cc:302] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f98dee122a6451e92ca7605195767a4; no voters:
I20260809 00:36:47.088047 9677 leader_election.cc:288] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:47.088117 9680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:47.088297 9680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [term 1 LEADER]: Becoming Leader. State: Replica: 2f98dee122a6451e92ca7605195767a4, State: Running, Role: LEADER
I20260809 00:36:47.088315 9677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:47.088399 9680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f98dee122a6451e92ca7605195767a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260809 00:36:47.088692 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 9609
I20260809 00:36:47.088815 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:47.089062 21671 external_mini_cluster.cc:1428] Setting key eadfc68e53fe31b57694acc1fbfacb4b
I20260809 00:36:47.089568 9682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f98dee122a6451e92ca7605195767a4. Latest consensus state: current_term: 1 leader_uuid: "2f98dee122a6451e92ca7605195767a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f98dee122a6451e92ca7605195767a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } } }
I20260809 00:36:47.089671 9682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:47.089900 9681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f98dee122a6451e92ca7605195767a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f98dee122a6451e92ca7605195767a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } } }
I20260809 00:36:47.090060 9681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4 [sys.catalog]: This master's current role is: LEADER
W20260809 00:36:47.090528 9696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:36:47.090588 9696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:36:47.090648 9697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:47.090808 9697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:47.092716 9697 catalog_manager.cc:1347] Generated new cluster ID: 9140488d58684c71a9ceb53f6ff74ae9
I20260809 00:36:47.092772 9697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:47.095093 9698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:47.089713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52807 (local address 127.0.0.1:33115)
0809 00:36:47.089967 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:47.089976 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:47.090015 (+ 39us) server_negotiation.cc:407] Connection header received
0809 00:36:47.090240 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:47.090246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:47.090325 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:47.090505 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:47.090770 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:47.092197 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:47.093152 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:47.093580 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:47.093704 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:47.094465 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:47.094469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:47.094469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:47.094657 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:47.094688 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:47.094699 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:47.094798 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:47.094888 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 00:36:47.094918 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":70,"threads_started":1}
I20260809 00:36:47.096180 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:36:47.101300 9698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:47.097654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48091 (local address 127.0.0.1:33115)
0809 00:36:47.097692 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:47.097694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:47.098519 (+ 825us) server_negotiation.cc:407] Connection header received
0809 00:36:47.098592 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:47.098593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:47.098603 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:47.098649 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:47.098782 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:47.099264 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:47.100023 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:47.100259 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:47.100356 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:47.101035 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:47.101036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:47.101037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:47.101041 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:47.101056 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:47.101061 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:47.101113 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:47.101173 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 00:36:47.101184 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:36:47.111881 9697 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:47.112413 9697 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:47.116968 9697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f98dee122a6451e92ca7605195767a4: Generated new TSK 0
I20260809 00:36:47.117179 9697 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:47.127786 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 9609
W20260809 00:36:47.131801 9594 connection.cc:535] client connection to 127.0.0.1:33115 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33115): Connection reset by peer (error 104)
2026-08-09T00:36:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1346 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:47 dist-test-slave-zpfg krb5kdc[9708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:47 dist-test-slave-zpfg krb5kdc[9708](info): set up 2 sockets
Aug 09 00:36:47 dist-test-slave-zpfg krb5kdc[9708](info): commencing operation
krb5kdc: starting...
W20260809 00:36:48.180392 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:48 dist-test-slave-zpfg krb5kdc[9708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:36:48.305789 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33003
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:48.387167 9721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:48.387437 9721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:48.387480 9721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:48.388684 9721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:48.388760 9721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:48.388784 9721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:48.388805 9721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:48.388833 9721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:48.390362 9721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33003
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:48.390650 9721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:48.390898 9721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:48.395771 9729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:48.395828 9726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:48.395885 9727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:48.395920 9721 server_base.cc:1034] running on GCE node
I20260809 00:36:48.396243 9721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:48.396539 9721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:48.397712 9721 hybrid_clock.cc:648] HybridClock initialized: now 1786235808397687 us; error 58 us; skew 500 ppm
Aug 09 00:36:48 dist-test-slave-zpfg krb5kdc[9708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235808, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:48.407573 9721 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:36:48.408185 9721 webserver.cc:460] Webserver started at http://127.0.0.1:38545/ using document root <none> and password file <none>
I20260809 00:36:48.408449 9721 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:48.408521 9721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:48.408676 9721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:48.409750 9721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "5da498bcdda142fea9b791d5fe9eac9b"
format_stamp: "Formatted at 2026-08-09 00:36:48 on dist-test-slave-zpfg"
server_key: "34766273c3ce5b5ed03943ee1720177b"
server_key_iv: "6d45bbc60d2d35ee440566cbb796715a"
server_key_version: "encryptionkey@0"
I20260809 00:36:48.410125 9721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "5da498bcdda142fea9b791d5fe9eac9b"
format_stamp: "Formatted at 2026-08-09 00:36:48 on dist-test-slave-zpfg"
server_key: "34766273c3ce5b5ed03943ee1720177b"
server_key_iv: "6d45bbc60d2d35ee440566cbb796715a"
server_key_version: "encryptionkey@0"
I20260809 00:36:48.411580 9721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:36:48.412418 9736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:48.412606 9721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:48.412710 9721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "5da498bcdda142fea9b791d5fe9eac9b"
format_stamp: "Formatted at 2026-08-09 00:36:48 on dist-test-slave-zpfg"
server_key: "34766273c3ce5b5ed03943ee1720177b"
server_key_iv: "6d45bbc60d2d35ee440566cbb796715a"
server_key_version: "encryptionkey@0"
I20260809 00:36:48.412817 9721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:48.416294 9721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:48.418167 9721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:48.418329 9721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:48.422528 9721 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33003
I20260809 00:36:48.422564 9788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33003 every 8 connection(s)
I20260809 00:36:48.422915 9721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:48.423764 9789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:48.426010 9789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b: Bootstrap starting.
I20260809 00:36:48.426623 9789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:48.426875 9789 log.cc:824] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:48.427551 9789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b: No bootstrap required, opened a new log
I20260809 00:36:48.429394 9789 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da498bcdda142fea9b791d5fe9eac9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33003 } }
I20260809 00:36:48.429543 9789 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:48.429567 9789 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5da498bcdda142fea9b791d5fe9eac9b, State: Initialized, Role: FOLLOWER
I20260809 00:36:48.429790 9789 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da498bcdda142fea9b791d5fe9eac9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33003 } }
I20260809 00:36:48.429872 9789 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:48.429894 9789 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:48.429924 9789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:48.430522 9789 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da498bcdda142fea9b791d5fe9eac9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33003 } }
I20260809 00:36:48.430612 9789 leader_election.cc:302] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5da498bcdda142fea9b791d5fe9eac9b; no voters:
I20260809 00:36:48.430760 9789 leader_election.cc:288] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:48.430835 9792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:48.430968 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 9721
I20260809 00:36:48.431034 9792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [term 1 LEADER]: Becoming Leader. State: Replica: 5da498bcdda142fea9b791d5fe9eac9b, State: Running, Role: LEADER
I20260809 00:36:48.431058 9789 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:48.431083 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:48.431128 9792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da498bcdda142fea9b791d5fe9eac9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33003 } }
I20260809 00:36:48.431305 21671 external_mini_cluster.cc:1428] Setting key 1e5c4859e9e47174fa1369c43d0a3d51
I20260809 00:36:48.431821 9793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5da498bcdda142fea9b791d5fe9eac9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da498bcdda142fea9b791d5fe9eac9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33003 } } }
I20260809 00:36:48.431910 9793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:48.432423 9794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5da498bcdda142fea9b791d5fe9eac9b. Latest consensus state: current_term: 1 leader_uuid: "5da498bcdda142fea9b791d5fe9eac9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da498bcdda142fea9b791d5fe9eac9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33003 } } }
I20260809 00:36:48.432518 9794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b [sys.catalog]: This master's current role is: LEADER
W20260809 00:36:48.433133 9811 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:36:48.433189 9811 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:36:48.433243 9802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:48.433704 9802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:48.434273 9803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:48.431975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56469 (local address 127.0.0.1:33003)
0809 00:36:48.432752 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:48.432763 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:48.432800 (+ 37us) server_negotiation.cc:407] Connection header received
0809 00:36:48.432918 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:48.432923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:48.432947 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:48.433111 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:48.433842 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:48.433844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:48.433845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:48.433847 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:48.433874 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:48.433885 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:48.433965 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:48.434030 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 00:36:48.434070 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":70,"threads_started":1}
I20260809 00:36:48.435372 9802 catalog_manager.cc:1347] Generated new cluster ID: d7d5851f4a1d420c9fadfa202fccb1b2
I20260809 00:36:48.435397 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:36:48.435426 9802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:48.440099 9803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:48.438778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40973 (local address 127.0.0.1:33003)
0809 00:36:48.438821 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:48.438823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:48.439054 (+ 231us) server_negotiation.cc:407] Connection header received
0809 00:36:48.439122 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:48.439123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:48.439129 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:48.439185 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:48.439854 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:48.439855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:48.439856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:48.439860 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:48.439882 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:48.439886 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:48.439952 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:48.440003 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 00:36:48.440018 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:36:48.476341 9802 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:48.476553 9802 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:48.495784 9802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5da498bcdda142fea9b791d5fe9eac9b: Generated new TSK 0
I20260809 00:36:48.496018 9802 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:48.497721 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 9721
2026-08-09T00:36:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1370 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:48 dist-test-slave-zpfg krb5kdc[9821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:48 dist-test-slave-zpfg krb5kdc[9821](info): set up 2 sockets
Aug 09 00:36:48 dist-test-slave-zpfg krb5kdc[9821](info): commencing operation
krb5kdc: starting...
W20260809 00:36:49.552167 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:49 dist-test-slave-zpfg krb5kdc[9821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:36:49.673780 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34623
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:49.757050 9834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:49.757323 9834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:49.757361 9834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:49.758548 9834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:49.758622 9834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:49.758646 9834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:49.758670 9834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:49.758697 9834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:49.760217 9834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:49.760504 9834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:49.760744 9834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:49.765866 9839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:49.765875 9840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:49.765887 9834 server_base.cc:1034] running on GCE node
W20260809 00:36:49.765967 9842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:49.766285 9834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:49.766628 9834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:49.767799 9834 hybrid_clock.cc:648] HybridClock initialized: now 1786235809767764 us; error 46 us; skew 500 ppm
Aug 09 00:36:49 dist-test-slave-zpfg krb5kdc[9821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235809, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:49.770527 9834 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:36:49.771004 9834 webserver.cc:460] Webserver started at http://127.0.0.1:45521/ using document root <none> and password file <none>
I20260809 00:36:49.771193 9834 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:49.771232 9834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:49.771322 9834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:49.772336 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "ab00bb38cb4845968fa37447163fd465"
format_stamp: "Formatted at 2026-08-09 00:36:49 on dist-test-slave-zpfg"
server_key: "c967c749a4af2e93d2d89b382e3a6650"
server_key_iv: "f6799f0dc2a45b8815c7b7d1d3069910"
server_key_version: "encryptionkey@0"
I20260809 00:36:49.772650 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "ab00bb38cb4845968fa37447163fd465"
format_stamp: "Formatted at 2026-08-09 00:36:49 on dist-test-slave-zpfg"
server_key: "c967c749a4af2e93d2d89b382e3a6650"
server_key_iv: "f6799f0dc2a45b8815c7b7d1d3069910"
server_key_version: "encryptionkey@0"
I20260809 00:36:49.774086 9834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:36:49.774979 9849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:49.775180 9834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:36:49.775271 9834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "ab00bb38cb4845968fa37447163fd465"
format_stamp: "Formatted at 2026-08-09 00:36:49 on dist-test-slave-zpfg"
server_key: "c967c749a4af2e93d2d89b382e3a6650"
server_key_iv: "f6799f0dc2a45b8815c7b7d1d3069910"
server_key_version: "encryptionkey@0"
I20260809 00:36:49.775328 9834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:49.800328 9834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:49.802731 9834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:49.802850 9834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:49.807006 9834 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34623
I20260809 00:36:49.807078 9901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34623 every 8 connection(s)
I20260809 00:36:49.807425 9834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:49.808302 9902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:49.809425 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 9834
I20260809 00:36:49.809516 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:49.809720 21671 external_mini_cluster.cc:1428] Setting key e34ded638e8504b9f8f2b11204104c7a
I20260809 00:36:49.811180 9902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465: Bootstrap starting.
I20260809 00:36:49.811822 9902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:49.812126 9902 log.cc:824] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:49.812841 9902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465: No bootstrap required, opened a new log
I20260809 00:36:49.814661 9902 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab00bb38cb4845968fa37447163fd465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } }
I20260809 00:36:49.814819 9902 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:49.814865 9902 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab00bb38cb4845968fa37447163fd465, State: Initialized, Role: FOLLOWER
I20260809 00:36:49.814998 9902 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab00bb38cb4845968fa37447163fd465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } }
I20260809 00:36:49.815095 9902 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:49.815141 9902 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:49.815191 9902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:49.815779 9902 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab00bb38cb4845968fa37447163fd465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } }
I20260809 00:36:49.815900 9902 leader_election.cc:302] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab00bb38cb4845968fa37447163fd465; no voters:
I20260809 00:36:49.816084 9902 leader_election.cc:288] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:49.816128 9907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:49.816330 9907 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [term 1 LEADER]: Becoming Leader. State: Replica: ab00bb38cb4845968fa37447163fd465, State: Running, Role: LEADER
I20260809 00:36:49.816349 9902 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 00:36:49 dist-test-slave-zpfg krb5kdc[9821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 00:36:49.816438 9907 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab00bb38cb4845968fa37447163fd465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } }
I20260809 00:36:49.816756 9907 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab00bb38cb4845968fa37447163fd465. Latest consensus state: current_term: 1 leader_uuid: "ab00bb38cb4845968fa37447163fd465" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab00bb38cb4845968fa37447163fd465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } } }
I20260809 00:36:49.816833 9907 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:49.816769 9908 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab00bb38cb4845968fa37447163fd465" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab00bb38cb4845968fa37447163fd465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } } }
I20260809 00:36:49.817008 9908 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:49.817370 9914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:49.818262 9914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:49.820173 9914 catalog_manager.cc:1347] Generated new cluster ID: 10abbcd51e2e43be91e3bb3857ff1baf
I20260809 00:36:49.820232 9914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:49.821065 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:49.810309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33943 (local address 127.0.0.1:34623)
0809 00:36:49.810555 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:49.810565 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:49.810603 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:36:49.811157 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:49.811166 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:49.811230 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:49.811380 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:49.811995 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:49.812960 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:49.813723 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:49.814135 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:49.814254 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:49.816994 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:49.816998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:49.816998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:49.817011 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:49.819099 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:49.819545 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:49.819548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:49.819549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:49.819664 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:49.819869 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:49.819871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:49.819871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:49.820151 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:49.820295 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:49.820715 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 00:36:49.820848 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":141,"threads_started":1}
I20260809 00:36:49.822057 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:36:49.831194 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:49.823621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60633 (local address 127.0.0.1:34623)
0809 00:36:49.823662 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:49.823664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:49.823673 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:36:49.823732 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:49.823732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:49.823741 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:49.823815 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:49.824427 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:49.825518 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:49.826520 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:49.826872 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:49.826946 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:49.828285 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:49.828287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:49.828287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:49.828294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:49.829848 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:49.830258 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:49.830259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:49.830259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:49.830319 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:49.830551 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:49.830552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:49.830552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:49.830686 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:49.830799 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:49.830912 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 00:36:49.831003 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:36:49.840616 9914 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:49.841355 9914 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:49.850426 9914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab00bb38cb4845968fa37447163fd465: Generated new TSK 0
I20260809 00:36:49.850673 9914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:49.860155 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 9834
2026-08-09T00:36:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:49 dist-test-slave-zpfg krb5kdc[9933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:49 dist-test-slave-zpfg krb5kdc[9933](info): set up 2 sockets
Aug 09 00:36:49 dist-test-slave-zpfg krb5kdc[9933](info): commencing operation
krb5kdc: starting...
W20260809 00:36:50.920146 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:51 dist-test-slave-zpfg krb5kdc[9933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:36:51.053328 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45843
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:51.135380 9946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:51.135653 9946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:51.135692 9946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:51.136829 9946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:51.136901 9946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:51.136925 9946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:51.136947 9946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:51.136976 9946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:51.138485 9946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:51.138765 9946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:51.138996 9946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:36:51.144052 9946 server_base.cc:1034] running on GCE node
W20260809 00:36:51.144016 9952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:51.144021 9951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:51.144003 9954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:51.144588 9946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:51.144840 9946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:51.146005 9946 hybrid_clock.cc:648] HybridClock initialized: now 1786235811145993 us; error 30 us; skew 500 ppm
Aug 09 00:36:51 dist-test-slave-zpfg krb5kdc[9933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235811, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:51.148660 9946 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:36:51.149135 9946 webserver.cc:460] Webserver started at http://127.0.0.1:40975/ using document root <none> and password file <none>
I20260809 00:36:51.149325 9946 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:51.149369 9946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:51.149458 9946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:51.150545 9946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "dcff5e3c91544ee7b7877d31d0a370d5"
format_stamp: "Formatted at 2026-08-09 00:36:51 on dist-test-slave-zpfg"
server_key: "d136b3d5ea93a22bdd98da979bc24d34"
server_key_iv: "fcaf5d93dbf5b57a5073268b8d4b9b59"
server_key_version: "encryptionkey@0"
I20260809 00:36:51.150861 9946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "dcff5e3c91544ee7b7877d31d0a370d5"
format_stamp: "Formatted at 2026-08-09 00:36:51 on dist-test-slave-zpfg"
server_key: "d136b3d5ea93a22bdd98da979bc24d34"
server_key_iv: "fcaf5d93dbf5b57a5073268b8d4b9b59"
server_key_version: "encryptionkey@0"
I20260809 00:36:51.152207 9946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:51.152900 9961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:51.153090 9946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:36:51.153182 9946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "dcff5e3c91544ee7b7877d31d0a370d5"
format_stamp: "Formatted at 2026-08-09 00:36:51 on dist-test-slave-zpfg"
server_key: "d136b3d5ea93a22bdd98da979bc24d34"
server_key_iv: "fcaf5d93dbf5b57a5073268b8d4b9b59"
server_key_version: "encryptionkey@0"
I20260809 00:36:51.153256 9946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:51.185267 9946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:51.187944 9946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:51.188103 9946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:51.192267 9946 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45843
I20260809 00:36:51.192342 10013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45843 every 8 connection(s)
I20260809 00:36:51.192690 9946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:51.193584 10014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:51.196111 10014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5: Bootstrap starting.
I20260809 00:36:51.196712 10014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:51.196993 10014 log.cc:824] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:51.197711 10014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5: No bootstrap required, opened a new log
I20260809 00:36:51.198804 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 9946
I20260809 00:36:51.198951 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:51.199156 21671 external_mini_cluster.cc:1428] Setting key fb1c99ffc0b98801f7b2f0bdb1e8671e
I20260809 00:36:51.199651 10014 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcff5e3c91544ee7b7877d31d0a370d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45843 } }
I20260809 00:36:51.199911 10014 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:51.199956 10014 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcff5e3c91544ee7b7877d31d0a370d5, State: Initialized, Role: FOLLOWER
I20260809 00:36:51.200101 10014 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcff5e3c91544ee7b7877d31d0a370d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45843 } }
I20260809 00:36:51.200188 10014 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:51.200212 10014 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:51.200304 10014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:51.201212 10014 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcff5e3c91544ee7b7877d31d0a370d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45843 } }
I20260809 00:36:51.201345 10014 leader_election.cc:302] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcff5e3c91544ee7b7877d31d0a370d5; no voters:
I20260809 00:36:51.201510 10014 leader_election.cc:288] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:51.201582 10019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:51.201726 10019 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [term 1 LEADER]: Becoming Leader. State: Replica: dcff5e3c91544ee7b7877d31d0a370d5, State: Running, Role: LEADER
I20260809 00:36:51.201830 10014 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:51.201869 10019 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcff5e3c91544ee7b7877d31d0a370d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45843 } }
I20260809 00:36:51.202201 10019 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcff5e3c91544ee7b7877d31d0a370d5. Latest consensus state: current_term: 1 leader_uuid: "dcff5e3c91544ee7b7877d31d0a370d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcff5e3c91544ee7b7877d31d0a370d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45843 } } }
I20260809 00:36:51.202297 10019 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:51.202211 10021 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcff5e3c91544ee7b7877d31d0a370d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcff5e3c91544ee7b7877d31d0a370d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45843 } } }
I20260809 00:36:51.202451 10021 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:51.202560 10026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:51.203066 10026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:51.205010 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:51.199675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38371 (local address 127.0.0.1:45843)
0809 00:36:51.199903 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:51.199914 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:51.199952 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:36:51.200122 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:51.200154 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:51.200227 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:51.200377 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:51.200573 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:51.201461 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:51.202703 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:51.203309 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:51.203441 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:51.204218 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:51.204221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:51.204222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:51.204451 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:51.204490 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:51.204505 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:51.204579 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:51.204703 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 00:36:51.204789 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":107,"threads_started":1}
I20260809 00:36:51.205214 10026 catalog_manager.cc:1347] Generated new cluster ID: 883dfd2bde184157b34a48393cb7c7c6
I20260809 00:36:51.205260 10026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:51.205955 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:36:51.211012 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:51.207414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40301 (local address 127.0.0.1:45843)
0809 00:36:51.207446 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:51.207448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:51.207579 (+ 131us) server_negotiation.cc:407] Connection header received
0809 00:36:51.207641 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:51.207642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:51.207666 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:51.207721 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:51.207874 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:51.208571 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:51.209599 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:51.209897 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:51.209955 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:51.210746 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:51.210747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:51.210747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:51.210751 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:51.210765 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:51.210770 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:51.210828 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:51.210909 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 00:36:51.210934 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 00:36:51.215771 10026 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:51.216303 10026 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:51.220940 10026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcff5e3c91544ee7b7877d31d0a370d5: Generated new TSK 0
I20260809 00:36:51.221131 10026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:51.224865 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 9946
2026-08-09T00:36:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1364 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:51 dist-test-slave-zpfg krb5kdc[10045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:51 dist-test-slave-zpfg krb5kdc[10045](info): set up 2 sockets
Aug 09 00:36:51 dist-test-slave-zpfg krb5kdc[10045](info): commencing operation
krb5kdc: starting...
W20260809 00:36:52.260309 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:52 dist-test-slave-zpfg krb5kdc[10045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:36:52.382534 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42673
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:52.465090 10058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:52.465293 10058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:52.465317 10058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:52.466420 10058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:52.466455 10058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:52.466470 10058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:52.466482 10058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:52.466495 10058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:52.467986 10058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:52.468180 10058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:52.468360 10058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:52.473212 10064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:52.473242 10063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:52.473242 10066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:52.473495 10058 server_base.cc:1034] running on GCE node
I20260809 00:36:52.473762 10058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:52.474068 10058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:52.475239 10058 hybrid_clock.cc:648] HybridClock initialized: now 1786235812475226 us; error 46 us; skew 500 ppm
Aug 09 00:36:52 dist-test-slave-zpfg krb5kdc[10045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235812, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:52.478091 10058 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:36:52.478602 10058 webserver.cc:460] Webserver started at http://127.0.0.1:38665/ using document root <none> and password file <none>
I20260809 00:36:52.478857 10058 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:52.478930 10058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:52.479077 10058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:52.480156 10058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "9627fd4d4af7462cab03df75bfe58d58"
format_stamp: "Formatted at 2026-08-09 00:36:52 on dist-test-slave-zpfg"
server_key: "fa7ded37f96495286902481f6da76773"
server_key_iv: "66106865a31b90f841943f494f053248"
server_key_version: "encryptionkey@0"
I20260809 00:36:52.480535 10058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "9627fd4d4af7462cab03df75bfe58d58"
format_stamp: "Formatted at 2026-08-09 00:36:52 on dist-test-slave-zpfg"
server_key: "fa7ded37f96495286902481f6da76773"
server_key_iv: "66106865a31b90f841943f494f053248"
server_key_version: "encryptionkey@0"
I20260809 00:36:52.481876 10058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:36:52.482681 10073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:52.482860 10058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:36:52.482972 10058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "9627fd4d4af7462cab03df75bfe58d58"
format_stamp: "Formatted at 2026-08-09 00:36:52 on dist-test-slave-zpfg"
server_key: "fa7ded37f96495286902481f6da76773"
server_key_iv: "66106865a31b90f841943f494f053248"
server_key_version: "encryptionkey@0"
I20260809 00:36:52.483050 10058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:52.486290 10058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:52.488682 10058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:52.488850 10058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:52.492664 10058 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42673
I20260809 00:36:52.492715 10125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42673 every 8 connection(s)
I20260809 00:36:52.493047 10058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:52.494045 10126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:52.496275 10126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58: Bootstrap starting.
I20260809 00:36:52.496806 10126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:52.497041 10126 log.cc:824] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:52.497434 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 10058
I20260809 00:36:52.497557 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:52.497761 21671 external_mini_cluster.cc:1428] Setting key d057c71dd34ebf0243286235478d4d59
I20260809 00:36:52.497795 10126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58: No bootstrap required, opened a new log
I20260809 00:36:52.499655 10126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9627fd4d4af7462cab03df75bfe58d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42673 } }
I20260809 00:36:52.499830 10126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:52.499874 10126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9627fd4d4af7462cab03df75bfe58d58, State: Initialized, Role: FOLLOWER
I20260809 00:36:52.500061 10126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9627fd4d4af7462cab03df75bfe58d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42673 } }
I20260809 00:36:52.500166 10126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:52.500191 10126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:52.500248 10126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:52.501006 10130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:52.498274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51011 (local address 127.0.0.1:42673)
0809 00:36:52.498470 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:52.498481 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:52.498526 (+ 45us) server_negotiation.cc:407] Connection header received
0809 00:36:52.499636 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:52.499642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:52.499669 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:52.499786 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:52.500515 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:52.500518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:52.500518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:52.500522 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:52.500561 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:52.500571 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:52.500627 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:52.500726 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 00:36:52.500757 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
I20260809 00:36:52.501325 10126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9627fd4d4af7462cab03df75bfe58d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42673 } }
I20260809 00:36:52.501452 10126 leader_election.cc:302] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9627fd4d4af7462cab03df75bfe58d58; no voters:
I20260809 00:36:52.501610 10126 leader_election.cc:288] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:52.501639 10131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:52.501783 10131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [term 1 LEADER]: Becoming Leader. State: Replica: 9627fd4d4af7462cab03df75bfe58d58, State: Running, Role: LEADER
I20260809 00:36:52.501855 10126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:52.501925 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:36:52.501971 10131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9627fd4d4af7462cab03df75bfe58d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42673 } }
I20260809 00:36:52.502282 10131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9627fd4d4af7462cab03df75bfe58d58. Latest consensus state: current_term: 1 leader_uuid: "9627fd4d4af7462cab03df75bfe58d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9627fd4d4af7462cab03df75bfe58d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42673 } } }
I20260809 00:36:52.502369 10131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:52.502526 10132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9627fd4d4af7462cab03df75bfe58d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9627fd4d4af7462cab03df75bfe58d58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42673 } } }
I20260809 00:36:52.502651 10132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58 [sys.catalog]: This master's current role is: LEADER
W20260809 00:36:52.503746 10148 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:36:52.503855 10148 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:36:52.503973 10138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:52.504199 10138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:52.504873 10130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:52.503702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46461 (local address 127.0.0.1:42673)
0809 00:36:52.503737 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:52.503740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:52.503876 (+ 136us) server_negotiation.cc:407] Connection header received
0809 00:36:52.503982 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:52.503983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:52.503988 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:52.504045 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:52.504660 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:52.504661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:52.504661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:52.504666 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:52.504679 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:52.504683 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:52.504732 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:52.504795 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 00:36:52.504810 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 00:36:52.505786 10138 catalog_manager.cc:1347] Generated new cluster ID: 8ef1c36102bf45c39753fc28727c87d3
I20260809 00:36:52.505841 10138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:52.512617 10138 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:52.512770 10138 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:52.519438 10138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9627fd4d4af7462cab03df75bfe58d58: Generated new TSK 0
I20260809 00:36:52.519627 10138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:52.526468 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 10058
2026-08-09T00:36:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:52 dist-test-slave-zpfg krb5kdc[10157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:52 dist-test-slave-zpfg krb5kdc[10157](info): set up 2 sockets
Aug 09 00:36:52 dist-test-slave-zpfg krb5kdc[10157](info): commencing operation
krb5kdc: starting...
W20260809 00:36:53.624270 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:53 dist-test-slave-zpfg krb5kdc[10157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:36:53.745427 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37975
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:53.826622 10170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:53.826896 10170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:53.826939 10170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:53.828143 10170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:53.828220 10170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:53.828244 10170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:53.828265 10170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:53.828294 10170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:53.829900 10170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:53.830201 10170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:53.830471 10170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:53.835395 10178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:53.835431 10175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:53.835517 10176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:53.835564 10170 server_base.cc:1034] running on GCE node
I20260809 00:36:53.835925 10170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:53.836179 10170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:53.837327 10170 hybrid_clock.cc:648] HybridClock initialized: now 1786235813837314 us; error 32 us; skew 500 ppm
Aug 09 00:36:53 dist-test-slave-zpfg krb5kdc[10157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:53.845822 10170 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:36:53.846405 10170 webserver.cc:460] Webserver started at http://127.0.0.1:40607/ using document root <none> and password file <none>
I20260809 00:36:53.846654 10170 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:53.846729 10170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:53.846906 10170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:53.848028 10170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "31d52828b99543e1a1677a253ecbc47b"
format_stamp: "Formatted at 2026-08-09 00:36:53 on dist-test-slave-zpfg"
server_key: "7438216e75343eee79c326bbb30abe01"
server_key_iv: "6557735e41b76374e5bacd8d34481a39"
server_key_version: "encryptionkey@0"
I20260809 00:36:53.848420 10170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "31d52828b99543e1a1677a253ecbc47b"
format_stamp: "Formatted at 2026-08-09 00:36:53 on dist-test-slave-zpfg"
server_key: "7438216e75343eee79c326bbb30abe01"
server_key_iv: "6557735e41b76374e5bacd8d34481a39"
server_key_version: "encryptionkey@0"
I20260809 00:36:53.849826 10170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:36:53.850656 10185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:53.850868 10170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:53.850983 10170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "31d52828b99543e1a1677a253ecbc47b"
format_stamp: "Formatted at 2026-08-09 00:36:53 on dist-test-slave-zpfg"
server_key: "7438216e75343eee79c326bbb30abe01"
server_key_iv: "6557735e41b76374e5bacd8d34481a39"
server_key_version: "encryptionkey@0"
I20260809 00:36:53.851061 10170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:53.889780 10170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:53.892284 10170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:53.892472 10170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:53.896720 10170 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37975
I20260809 00:36:53.896757 10237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37975 every 8 connection(s)
I20260809 00:36:53.897146 10170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:53.898005 10238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:53.900380 10238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b: Bootstrap starting.
I20260809 00:36:53.900844 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 10170
I20260809 00:36:53.900974 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:53.900965 10238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:53.901177 21671 external_mini_cluster.cc:1428] Setting key 5e120b445f1e14c453e90c919920942b
I20260809 00:36:53.901304 10238 log.cc:824] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:53.902153 10238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b: No bootstrap required, opened a new log
I20260809 00:36:53.904655 10238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d52828b99543e1a1677a253ecbc47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } }
I20260809 00:36:53.904837 10238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:53.904877 10238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d52828b99543e1a1677a253ecbc47b, State: Initialized, Role: FOLLOWER
I20260809 00:36:53.905004 10238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d52828b99543e1a1677a253ecbc47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } }
I20260809 00:36:53.905076 10238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:53.905104 10238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:53.905146 10238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:53.905802 10238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d52828b99543e1a1677a253ecbc47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } }
I20260809 00:36:53.905893 10238 leader_election.cc:302] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31d52828b99543e1a1677a253ecbc47b; no voters:
I20260809 00:36:53.906162 10238 leader_election.cc:288] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:53.906183 10243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:53.906306 10243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [term 1 LEADER]: Becoming Leader. State: Replica: 31d52828b99543e1a1677a253ecbc47b, State: Running, Role: LEADER
I20260809 00:36:53.906476 10238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:53.906435 10243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d52828b99543e1a1677a253ecbc47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } }
I20260809 00:36:53.907938 10245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31d52828b99543e1a1677a253ecbc47b. Latest consensus state: current_term: 1 leader_uuid: "31d52828b99543e1a1677a253ecbc47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d52828b99543e1a1677a253ecbc47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } } }
I20260809 00:36:53.908313 10245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [sys.catalog]: This master's current role is: LEADER
Aug 09 00:36:53 dist-test-slave-zpfg krb5kdc[10157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
W20260809 00:36:53.908458 10258 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:36:53.908501 10258 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:36:53.908110 10244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31d52828b99543e1a1677a253ecbc47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d52828b99543e1a1677a253ecbc47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } } }
I20260809 00:36:53.908934 10244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:53.909335 10260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:53.909524 10260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:53.911108 10260 catalog_manager.cc:1347] Generated new cluster ID: a8dc085906e541309b63d401d9a433d1
I20260809 00:36:53.911159 10260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:53.912889 10242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:53.901703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:60475 (local address 127.0.0.1:37975)
0809 00:36:53.901905 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:53.901921 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:53.901950 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:36:53.902515 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:53.902523 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:53.902575 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:53.902688 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:53.903517 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:53.904436 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:53.905476 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:53.905908 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:53.906014 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:53.908928 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:53.908932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:53.908933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:53.908946 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:53.911117 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:53.911617 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:53.911620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:53.911621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:53.911724 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:53.911991 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:53.911992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:53.911993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:53.912277 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:53.912426 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:53.912571 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 00:36:53.912668 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":96,"threads_started":1}
I20260809 00:36:53.913898 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:36:53.921289 10260 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:53.921958 10260 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:53.922919 10242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:53.915346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:57055 (local address 127.0.0.1:37975)
0809 00:36:53.915384 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:53.915385 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:53.915427 (+ 42us) server_negotiation.cc:407] Connection header received
0809 00:36:53.915483 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:53.915484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:53.915493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:53.915555 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:53.916137 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:53.916912 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:53.917933 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:53.918255 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:53.918334 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:53.919788 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:53.919789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:53.919790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:53.919797 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:53.921545 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:53.922004 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:53.922005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:53.922005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:53.922069 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:53.922303 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:53.922304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:53.922304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:53.922468 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:53.922577 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:53.922746 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 00:36:53.922825 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:36:53.926410 10260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31d52828b99543e1a1677a253ecbc47b: Generated new TSK 0
I20260809 00:36:53.926594 10260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:53.929737 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 10170
W20260809 00:36:53.934212 10155 connection.cc:535] client connection to 127.0.0.1:37975 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37975): Connection reset by peer (error 104)
2026-08-09T00:36:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:53 dist-test-slave-zpfg krb5kdc[10269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:53 dist-test-slave-zpfg krb5kdc[10269](info): set up 2 sockets
Aug 09 00:36:53 dist-test-slave-zpfg krb5kdc[10269](info): commencing operation
krb5kdc: starting...
W20260809 00:36:54.968180 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:55 dist-test-slave-zpfg krb5kdc[10269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:36:55.091478 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44211
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:55.174311 10282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:55.174659 10282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:55.174700 10282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:55.175875 10282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:55.175947 10282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:55.175971 10282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:55.175992 10282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:55.176021 10282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:55.177484 10282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:55.177757 10282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:55.177986 10282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:55.182848 10288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:55.182865 10287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:55.182924 10282 server_base.cc:1034] running on GCE node
W20260809 00:36:55.182886 10290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:55.183352 10282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:55.183605 10282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:55.184748 10282 hybrid_clock.cc:648] HybridClock initialized: now 1786235815184732 us; error 29 us; skew 500 ppm
Aug 09 00:36:55 dist-test-slave-zpfg krb5kdc[10269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235815, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:55.187323 10282 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:36:55.187800 10282 webserver.cc:460] Webserver started at http://127.0.0.1:43423/ using document root <none> and password file <none>
I20260809 00:36:55.187996 10282 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:55.188036 10282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:55.188123 10282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:55.189152 10282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "f9fd76c7e776412dbfc519cc87046c51"
format_stamp: "Formatted at 2026-08-09 00:36:55 on dist-test-slave-zpfg"
server_key: "cf84a2922bf144ee16da980a89c12309"
server_key_iv: "dd8092ffdce744f23fca1c4162729a73"
server_key_version: "encryptionkey@0"
I20260809 00:36:55.189455 10282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "f9fd76c7e776412dbfc519cc87046c51"
format_stamp: "Formatted at 2026-08-09 00:36:55 on dist-test-slave-zpfg"
server_key: "cf84a2922bf144ee16da980a89c12309"
server_key_iv: "dd8092ffdce744f23fca1c4162729a73"
server_key_version: "encryptionkey@0"
I20260809 00:36:55.190804 10282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:36:55.191543 10297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:55.191761 10282 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:55.191843 10282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "f9fd76c7e776412dbfc519cc87046c51"
format_stamp: "Formatted at 2026-08-09 00:36:55 on dist-test-slave-zpfg"
server_key: "cf84a2922bf144ee16da980a89c12309"
server_key_iv: "dd8092ffdce744f23fca1c4162729a73"
server_key_version: "encryptionkey@0"
I20260809 00:36:55.191901 10282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:55.217325 10282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:55.219722 10282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:55.219838 10282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:55.224052 10282 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44211
I20260809 00:36:55.224112 10349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44211 every 8 connection(s)
I20260809 00:36:55.224470 10282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:55.225370 10350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:55.227106 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 10282
I20260809 00:36:55.227258 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:55.227470 21671 external_mini_cluster.cc:1428] Setting key e5ae88b801db6ec43cf0b220a3eb0923
I20260809 00:36:55.228194 10350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51: Bootstrap starting.
I20260809 00:36:55.228819 10350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:55.229168 10350 log.cc:824] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:55.229929 10350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51: No bootstrap required, opened a new log
I20260809 00:36:55.232336 10350 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9fd76c7e776412dbfc519cc87046c51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44211 } }
I20260809 00:36:55.232546 10350 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:55.232590 10350 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9fd76c7e776412dbfc519cc87046c51, State: Initialized, Role: FOLLOWER
I20260809 00:36:55.232726 10350 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9fd76c7e776412dbfc519cc87046c51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44211 } }
I20260809 00:36:55.232786 10350 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:55.232807 10350 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:55.232862 10350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:55.233595 10350 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9fd76c7e776412dbfc519cc87046c51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44211 } }
I20260809 00:36:55.233731 10350 leader_election.cc:302] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9fd76c7e776412dbfc519cc87046c51; no voters:
I20260809 00:36:55.233932 10353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:55.228094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:55323 (local address 127.0.0.1:44211)
0809 00:36:55.228330 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:55.228338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:55.228367 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:36:55.228883 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:55.228891 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:55.228994 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:55.229137 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:55.229384 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:55.230634 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:55.231803 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:55.232244 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:55.232334 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:55.233112 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:55.233116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:55.233116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:55.233286 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:55.233318 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:55.233335 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:55.233413 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:55.233603 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 00:36:55.233638 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":173,"thread_start_us":138,"threads_started":1}
I20260809 00:36:55.233945 10350 leader_election.cc:288] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:55.233978 10355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:55.234120 10355 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [term 1 LEADER]: Becoming Leader. State: Replica: f9fd76c7e776412dbfc519cc87046c51, State: Running, Role: LEADER
I20260809 00:36:55.234236 10355 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9fd76c7e776412dbfc519cc87046c51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44211 } }
I20260809 00:36:55.234278 10350 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:55.234707 10356 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9fd76c7e776412dbfc519cc87046c51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9fd76c7e776412dbfc519cc87046c51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44211 } } }
I20260809 00:36:55.234972 10356 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:55.235143 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:36:55.234884 10357 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9fd76c7e776412dbfc519cc87046c51. Latest consensus state: current_term: 1 leader_uuid: "f9fd76c7e776412dbfc519cc87046c51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9fd76c7e776412dbfc519cc87046c51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44211 } } }
I20260809 00:36:55.235481 10365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:55.235486 10357 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:55.236110 10365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:55.237747 10365 catalog_manager.cc:1347] Generated new cluster ID: ab2f609d8de94f178e2608a5c2775180
I20260809 00:36:55.237803 10365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:55.240746 10353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:55.236846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:53953 (local address 127.0.0.1:44211)
0809 00:36:55.236888 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:55.236890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:55.237252 (+ 362us) server_negotiation.cc:407] Connection header received
0809 00:36:55.237360 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:55.237361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:55.237373 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:55.237424 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:55.237600 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:55.238331 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:55.239344 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:55.239642 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:55.239690 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:55.240476 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:55.240477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:55.240478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:55.240482 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:55.240495 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:55.240499 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:55.240539 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:55.240665 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 00:36:55.240674 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:36:55.248847 10365 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:55.249650 10365 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:55.258538 10365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9fd76c7e776412dbfc519cc87046c51: Generated new TSK 0
I20260809 00:36:55.258752 10365 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:55.264492 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 10282
W20260809 00:36:55.269270 10267 connection.cc:535] client connection to 127.0.0.1:44211 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44211): Connection reset by peer (error 104)
2026-08-09T00:36:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:55 dist-test-slave-zpfg krb5kdc[10381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:55 dist-test-slave-zpfg krb5kdc[10381](info): set up 2 sockets
Aug 09 00:36:55 dist-test-slave-zpfg krb5kdc[10381](info): commencing operation
krb5kdc: starting...
W20260809 00:36:56.308142 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:56 dist-test-slave-zpfg krb5kdc[10381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:36:56.431403 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45389
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:56.516642 10394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:56.516855 10394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:56.516878 10394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:56.518117 10394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:56.518170 10394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:56.518186 10394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:56.518199 10394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:56.518256 10394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:56.519786 10394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:56.520051 10394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:56.520290 10394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:56.525210 10402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:56.525276 10399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:56.525220 10400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:56.525312 10394 server_base.cc:1034] running on GCE node
I20260809 00:36:56.525673 10394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:56.525983 10394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:56.527191 10394 hybrid_clock.cc:648] HybridClock initialized: now 1786235816527172 us; error 47 us; skew 500 ppm
Aug 09 00:36:56 dist-test-slave-zpfg krb5kdc[10381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235816, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:56.529852 10394 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:36:56.530308 10394 webserver.cc:460] Webserver started at http://127.0.0.1:46013/ using document root <none> and password file <none>
I20260809 00:36:56.530592 10394 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:56.530668 10394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:56.530828 10394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:56.531985 10394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "eb535fc9a76a4e60ad375def932a1483"
format_stamp: "Formatted at 2026-08-09 00:36:56 on dist-test-slave-zpfg"
server_key: "b1a96f2a0ef1b5c27ea14917fdf7d77a"
server_key_iv: "c45b33624ebdb1e225be1afb69a1c4b1"
server_key_version: "encryptionkey@0"
I20260809 00:36:56.532361 10394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "eb535fc9a76a4e60ad375def932a1483"
format_stamp: "Formatted at 2026-08-09 00:36:56 on dist-test-slave-zpfg"
server_key: "b1a96f2a0ef1b5c27ea14917fdf7d77a"
server_key_iv: "c45b33624ebdb1e225be1afb69a1c4b1"
server_key_version: "encryptionkey@0"
I20260809 00:36:56.533782 10394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:36:56.534644 10409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:56.534873 10394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:36:56.534983 10394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "eb535fc9a76a4e60ad375def932a1483"
format_stamp: "Formatted at 2026-08-09 00:36:56 on dist-test-slave-zpfg"
server_key: "b1a96f2a0ef1b5c27ea14917fdf7d77a"
server_key_iv: "c45b33624ebdb1e225be1afb69a1c4b1"
server_key_version: "encryptionkey@0"
I20260809 00:36:56.535065 10394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:56.538599 10394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:56.540941 10394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:56.541119 10394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:56.544992 10394 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45389
I20260809 00:36:56.545040 10461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45389 every 8 connection(s)
I20260809 00:36:56.545388 10394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:56.546437 10462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:56.546654 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 10394
I20260809 00:36:56.546782 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:56.546999 21671 external_mini_cluster.cc:1428] Setting key 9b83450024db9fe8548b633dd7ddfd50
I20260809 00:36:56.549445 10465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:56.547545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:50657 (local address 127.0.0.1:45389)
0809 00:36:56.547731 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:56.547747 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:56.547773 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:36:56.548189 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:56.548194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:56.548218 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:56.548343 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:56.548968 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:56.548971 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:56.548972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:56.548975 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:56.549013 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:56.549023 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:56.549079 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:56.549171 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 00:36:56.549233 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260809 00:36:56.550012 10462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483: Bootstrap starting.
I20260809 00:36:56.550484 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:36:56.550751 10462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:56.551033 10462 log.cc:824] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:56.551853 10462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483: No bootstrap required, opened a new log
I20260809 00:36:56.552877 10465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:56.551911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:41087 (local address 127.0.0.1:45389)
0809 00:36:56.551980 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:56.551982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:56.552033 (+ 51us) server_negotiation.cc:407] Connection header received
0809 00:36:56.552083 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:56.552084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:56.552088 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:56.552156 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:56.552668 (+ 512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:56.552669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:56.552669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:56.552673 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:56.552688 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:56.552693 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:56.552762 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:56.552796 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 00:36:56.552808 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:36:56.554169 10462 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb535fc9a76a4e60ad375def932a1483" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45389 } }
I20260809 00:36:56.554303 10462 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:56.554329 10462 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb535fc9a76a4e60ad375def932a1483, State: Initialized, Role: FOLLOWER
I20260809 00:36:56.554512 10462 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb535fc9a76a4e60ad375def932a1483" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45389 } }
I20260809 00:36:56.554603 10462 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:56.554649 10462 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:56.554700 10462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:56.555244 10462 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb535fc9a76a4e60ad375def932a1483" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45389 } }
I20260809 00:36:56.555361 10462 leader_election.cc:302] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb535fc9a76a4e60ad375def932a1483; no voters:
I20260809 00:36:56.555531 10462 leader_election.cc:288] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:56.555598 10472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:56.555749 10472 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [term 1 LEADER]: Becoming Leader. State: Replica: eb535fc9a76a4e60ad375def932a1483, State: Running, Role: LEADER
I20260809 00:36:56.555789 10462 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:56.555891 10472 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb535fc9a76a4e60ad375def932a1483" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45389 } }
I20260809 00:36:56.556281 10473 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb535fc9a76a4e60ad375def932a1483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb535fc9a76a4e60ad375def932a1483" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45389 } } }
I20260809 00:36:56.556565 10473 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:56.556486 10474 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb535fc9a76a4e60ad375def932a1483. Latest consensus state: current_term: 1 leader_uuid: "eb535fc9a76a4e60ad375def932a1483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb535fc9a76a4e60ad375def932a1483" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45389 } } }
I20260809 00:36:56.556829 10474 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:56.556931 10481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:56.557351 10481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:56.558956 10481 catalog_manager.cc:1347] Generated new cluster ID: c2ad9b4ca66e446499be4cc057058ad6
I20260809 00:36:56.559010 10481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:56.581063 10481 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:56.581272 10481 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:56.592630 10481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb535fc9a76a4e60ad375def932a1483: Generated new TSK 0
I20260809 00:36:56.592902 10481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:56.600818 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 10394
2026-08-09T00:36:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:56 dist-test-slave-zpfg krb5kdc[10493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:56 dist-test-slave-zpfg krb5kdc[10493](info): set up 2 sockets
Aug 09 00:36:56 dist-test-slave-zpfg krb5kdc[10493](info): commencing operation
krb5kdc: starting...
W20260809 00:36:57.648252 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:57 dist-test-slave-zpfg krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:36:57.769690 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34951
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:57.851193 10506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:57.851415 10506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:57.851439 10506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:57.853109 10506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:57.853161 10506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:57.853175 10506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:57.853188 10506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:57.853200 10506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:57.854773 10506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34951
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:57.855044 10506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:57.855301 10506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:36:57.860486 10506 server_base.cc:1034] running on GCE node
W20260809 00:36:57.860409 10512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:57.860414 10511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:57.860455 10514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:57.860918 10506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:57.861182 10506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:57.862372 10506 hybrid_clock.cc:648] HybridClock initialized: now 1786235817862319 us; error 29 us; skew 500 ppm
Aug 09 00:36:57 dist-test-slave-zpfg krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235817, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:57.864923 10506 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:36:57.865381 10506 webserver.cc:460] Webserver started at http://127.0.0.1:37429/ using document root <none> and password file <none>
I20260809 00:36:57.865584 10506 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:57.865625 10506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:57.865721 10506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:57.866847 10506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "8686c7b0b7034a35b50e35e2928acbb7"
format_stamp: "Formatted at 2026-08-09 00:36:57 on dist-test-slave-zpfg"
server_key: "133103025ca082efda98415d4ddae45d"
server_key_iv: "60def19ef7f4a3153ca65726ea26af7b"
server_key_version: "encryptionkey@0"
I20260809 00:36:57.867169 10506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "8686c7b0b7034a35b50e35e2928acbb7"
format_stamp: "Formatted at 2026-08-09 00:36:57 on dist-test-slave-zpfg"
server_key: "133103025ca082efda98415d4ddae45d"
server_key_iv: "60def19ef7f4a3153ca65726ea26af7b"
server_key_version: "encryptionkey@0"
I20260809 00:36:57.868556 10506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:36:57.869306 10521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:57.869493 10506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:36:57.869585 10506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "8686c7b0b7034a35b50e35e2928acbb7"
format_stamp: "Formatted at 2026-08-09 00:36:57 on dist-test-slave-zpfg"
server_key: "133103025ca082efda98415d4ddae45d"
server_key_iv: "60def19ef7f4a3153ca65726ea26af7b"
server_key_version: "encryptionkey@0"
I20260809 00:36:57.869643 10506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:57.877586 10506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:57.880100 10506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:57.880228 10506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:57.884109 10506 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34951
I20260809 00:36:57.884159 10573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34951 every 8 connection(s)
I20260809 00:36:57.884498 10506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:57.885430 10574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:57.887883 10574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7: Bootstrap starting.
I20260809 00:36:57.888492 10574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:57.888792 10574 log.cc:824] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:57.889568 10574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7: No bootstrap required, opened a new log
I20260809 00:36:57.891258 10574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8686c7b0b7034a35b50e35e2928acbb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260809 00:36:57.891424 10574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:57.891471 10574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8686c7b0b7034a35b50e35e2928acbb7, State: Initialized, Role: FOLLOWER
I20260809 00:36:57.891638 10574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8686c7b0b7034a35b50e35e2928acbb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260809 00:36:57.891723 10574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:57.891767 10574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:57.891819 10574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:57.892359 10574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8686c7b0b7034a35b50e35e2928acbb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260809 00:36:57.892481 10574 leader_election.cc:302] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8686c7b0b7034a35b50e35e2928acbb7; no voters:
I20260809 00:36:57.892673 10574 leader_election.cc:288] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:57.892716 10577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:57.892925 10577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [term 1 LEADER]: Becoming Leader. State: Replica: 8686c7b0b7034a35b50e35e2928acbb7, State: Running, Role: LEADER
I20260809 00:36:57.892943 10574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:57.893013 10577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8686c7b0b7034a35b50e35e2928acbb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260809 00:36:57.893401 10579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8686c7b0b7034a35b50e35e2928acbb7. Latest consensus state: current_term: 1 leader_uuid: "8686c7b0b7034a35b50e35e2928acbb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8686c7b0b7034a35b50e35e2928acbb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } } }
I20260809 00:36:57.893489 10579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:57.893595 10578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8686c7b0b7034a35b50e35e2928acbb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8686c7b0b7034a35b50e35e2928acbb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } } }
I20260809 00:36:57.893666 10578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:57.894122 10586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:57.894685 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 10506
I20260809 00:36:57.894763 10586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:57.894815 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:57.895038 21671 external_mini_cluster.cc:1428] Setting key 391b2928768aa8c5f0b26b7767f0ce77
I20260809 00:36:57.896543 10586 catalog_manager.cc:1347] Generated new cluster ID: 8f7a8c37335e40f6b9a6aba835734343
I20260809 00:36:57.896593 10586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:36:57 dist-test-slave-zpfg krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 00:36:57.905684 10596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:57.895531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:50533 (local address 127.0.0.1:34951)
0809 00:36:57.895713 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:57.895723 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:57.895975 (+ 252us) server_negotiation.cc:407] Connection header received
0809 00:36:57.896141 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:57.896149 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:57.896206 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:57.896363 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:57.897058 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:57.897968 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:57.898858 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:57.899299 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:57.899437 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:57.902227 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:57.902231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:57.902232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:57.902244 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:57.904092 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:57.904493 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:57.904496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:57.904496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:57.904606 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:57.904860 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:57.904861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:57.904861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:57.905110 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:57.905297 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:57.905398 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 00:36:57.905500 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
I20260809 00:36:57.906687 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:36:57.910655 10586 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:57.911230 10586 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:57.918298 10596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:57.908126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:50807 (local address 127.0.0.1:34951)
0809 00:36:57.908163 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:57.908165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:57.910665 (+ 2500us) server_negotiation.cc:407] Connection header received
0809 00:36:57.910690 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:57.910691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:57.910702 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:57.910757 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:57.911430 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:57.912200 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:57.913231 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:57.913571 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:57.913629 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:57.915151 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:57.915153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:57.915153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:36:57.915160 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:57.916714 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:57.917145 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:57.917146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:57.917147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:57.917200 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:36:57.917473 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:36:57.917474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:36:57.917474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:36:57.917604 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:57.917696 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:57.918139 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 00:36:57.918202 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:36:57.924809 10586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8686c7b0b7034a35b50e35e2928acbb7: Generated new TSK 0
I20260809 00:36:57.925031 10586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:57.929251 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 10506
W20260809 00:36:57.933913 10491 connection.cc:535] client connection to 127.0.0.1:34951 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34951): Connection reset by peer (error 104)
2026-08-09T00:36:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:57 dist-test-slave-zpfg krb5kdc[10605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:57 dist-test-slave-zpfg krb5kdc[10605](info): set up 2 sockets
Aug 09 00:36:57 dist-test-slave-zpfg krb5kdc[10605](info): commencing operation
krb5kdc: starting...
W20260809 00:36:58.984815 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:36:59 dist-test-slave-zpfg krb5kdc[10605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:36:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:36:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:36:59.109045 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36689
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:36:59.191476 10618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:36:59.191682 10618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:36:59.191706 10618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:36:59.192806 10618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:36:59.192844 10618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:36:59.192859 10618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:36:59.192871 10618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:36:59.192883 10618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:36:59.194283 10618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36689
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:36:59.194509 10618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:36:59.194695 10618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:36:59.199620 10626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:59.199644 10618 server_base.cc:1034] running on GCE node
W20260809 00:36:59.199653 10624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:36:59.199796 10623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:36:59.200120 10618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:36:59.200419 10618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:36:59.201586 10618 hybrid_clock.cc:648] HybridClock initialized: now 1786235819201574 us; error 45 us; skew 500 ppm
Aug 09 00:36:59 dist-test-slave-zpfg krb5kdc[10605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:36:59.208278 10618 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:36:59.208796 10618 webserver.cc:460] Webserver started at http://127.0.0.1:33325/ using document root <none> and password file <none>
I20260809 00:36:59.209088 10618 fs_manager.cc:359] Metadata directory not provided
I20260809 00:36:59.209160 10618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:36:59.209326 10618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:36:59.210484 10618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "d9048bb6799446b5937f725a2c1eed11"
format_stamp: "Formatted at 2026-08-09 00:36:59 on dist-test-slave-zpfg"
server_key: "c549f9e4f702662dfb6add0e8b8b3763"
server_key_iv: "c673f6d49f2843aff6b4daa500b29aa6"
server_key_version: "encryptionkey@0"
I20260809 00:36:59.210853 10618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "d9048bb6799446b5937f725a2c1eed11"
format_stamp: "Formatted at 2026-08-09 00:36:59 on dist-test-slave-zpfg"
server_key: "c549f9e4f702662dfb6add0e8b8b3763"
server_key_iv: "c673f6d49f2843aff6b4daa500b29aa6"
server_key_version: "encryptionkey@0"
I20260809 00:36:59.212230 10618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:36:59.213063 10633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:36:59.213289 10618 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:36:59.213425 10618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "d9048bb6799446b5937f725a2c1eed11"
format_stamp: "Formatted at 2026-08-09 00:36:59 on dist-test-slave-zpfg"
server_key: "c549f9e4f702662dfb6add0e8b8b3763"
server_key_iv: "c673f6d49f2843aff6b4daa500b29aa6"
server_key_version: "encryptionkey@0"
I20260809 00:36:59.213512 10618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:36:59.236909 10618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:36:59.239095 10618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:36:59.239285 10618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:36:59.243527 10618 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36689
I20260809 00:36:59.243566 10685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36689 every 8 connection(s)
I20260809 00:36:59.243973 10618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:36:59.244356 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 10618
I20260809 00:36:59.244495 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:36:59.244720 21671 external_mini_cluster.cc:1428] Setting key ef63d3cedd284c07d140f724a1a11d49
I20260809 00:36:59.244992 10686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:36:59.248158 10686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11: Bootstrap starting.
I20260809 00:36:59.248895 10686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11: Neither blocks nor log segments found. Creating new log.
I20260809 00:36:59.249255 10686 log.cc:824] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11: Log is configured to *not* fsync() on all Append() calls
I20260809 00:36:59.250068 10686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11: No bootstrap required, opened a new log
I20260809 00:36:59.250612 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:59.245296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:54959 (local address 127.0.0.1:36689)
0809 00:36:59.245520 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:59.245530 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:59.245567 (+ 37us) server_negotiation.cc:407] Connection header received
0809 00:36:59.246136 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:59.246144 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:59.246196 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:59.246377 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:59.246622 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:59.247559 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:59.248370 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:59.249034 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:59.249167 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:59.249905 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:59.249911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:59.249916 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:59.250076 (+ 160us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:59.250107 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:59.250117 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:59.250215 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:59.250327 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 00:36:59.250392 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
I20260809 00:36:59.251698 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:36:59.252730 10686 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9048bb6799446b5937f725a2c1eed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36689 } }
I20260809 00:36:59.252948 10686 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:36:59.253052 10686 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9048bb6799446b5937f725a2c1eed11, State: Initialized, Role: FOLLOWER
I20260809 00:36:59.253214 10686 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9048bb6799446b5937f725a2c1eed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36689 } }
I20260809 00:36:59.253300 10686 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:36:59.253326 10686 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:36:59.253404 10686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:36:59.254272 10686 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9048bb6799446b5937f725a2c1eed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36689 } }
I20260809 00:36:59.254402 10686 leader_election.cc:302] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9048bb6799446b5937f725a2c1eed11; no voters:
I20260809 00:36:59.254577 10686 leader_election.cc:288] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:36:59.254786 10696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:36:59.255028 10696 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [term 1 LEADER]: Becoming Leader. State: Replica: d9048bb6799446b5937f725a2c1eed11, State: Running, Role: LEADER
I20260809 00:36:59.255054 10686 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:36:59.255188 10696 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9048bb6799446b5937f725a2c1eed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36689 } }
I20260809 00:36:59.255595 10696 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9048bb6799446b5937f725a2c1eed11. Latest consensus state: current_term: 1 leader_uuid: "d9048bb6799446b5937f725a2c1eed11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9048bb6799446b5937f725a2c1eed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36689 } } }
I20260809 00:36:59.255877 10696 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:59.255802 10697 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9048bb6799446b5937f725a2c1eed11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9048bb6799446b5937f725a2c1eed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36689 } } }
I20260809 00:36:59.256319 10697 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11 [sys.catalog]: This master's current role is: LEADER
I20260809 00:36:59.256548 10705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:36:59.257146 10705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:36:59.257344 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:36:59.253421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:37213 (local address 127.0.0.1:36689)
0809 00:36:59.253466 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:36:59.253468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:36:59.253606 (+ 138us) server_negotiation.cc:407] Connection header received
0809 00:36:59.253634 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:36:59.253635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:36:59.253646 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:36:59.253717 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:36:59.253814 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:59.254572 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:59.255674 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:36:59.255977 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:36:59.256028 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:36:59.256940 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:36:59.256941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:36:59.256942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:36:59.256947 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:36:59.256962 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:36:59.256967 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:36:59.257022 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:36:59.257143 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 00:36:59.257154 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:36:59.258917 10705 catalog_manager.cc:1347] Generated new cluster ID: 314c249a7077429a946d60fb97d2025b
I20260809 00:36:59.258975 10705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:36:59.270879 10705 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:36:59.271512 10705 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:36:59.278623 10705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9048bb6799446b5937f725a2c1eed11: Generated new TSK 0
I20260809 00:36:59.278854 10705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:36:59.288763 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 10618
W20260809 00:36:59.293444 10603 connection.cc:535] client connection to 127.0.0.1:36689 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36689): Connection reset by peer (error 104)
2026-08-09T00:36:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1359 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:36:59 dist-test-slave-zpfg krb5kdc[10717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:36:59 dist-test-slave-zpfg krb5kdc[10717](info): set up 2 sockets
Aug 09 00:36:59 dist-test-slave-zpfg krb5kdc[10717](info): commencing operation
krb5kdc: starting...
W20260809 00:37:00.344172 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:37:00 dist-test-slave-zpfg krb5kdc[10717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:37:00.465337 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34857
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:00.547015 10730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:00.547266 10730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:00.547302 10730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:00.548419 10730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:00.548489 10730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:00.548516 10730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:00.548547 10730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:00.548570 10730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:00.550014 10730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34857
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:00.550274 10730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:00.550522 10730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:00.555238 10736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:00.555308 10735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:00.555497 10730 server_base.cc:1034] running on GCE node
W20260809 00:37:00.555255 10738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:00.555738 10730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:00.556053 10730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:00.557248 10730 hybrid_clock.cc:648] HybridClock initialized: now 1786235820557238 us; error 34 us; skew 500 ppm
Aug 09 00:37:00 dist-test-slave-zpfg krb5kdc[10717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235820, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:00.559796 10730 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:37:00.560310 10730 webserver.cc:460] Webserver started at http://127.0.0.1:33753/ using document root <none> and password file <none>
I20260809 00:37:00.560564 10730 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:00.560633 10730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:00.560788 10730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:00.561865 10730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "e458f0ade5934a0c97c8bd789c8b92d9"
format_stamp: "Formatted at 2026-08-09 00:37:00 on dist-test-slave-zpfg"
server_key: "966d9b3fe437acea8630cf4cfa1a318e"
server_key_iv: "7fbc5ba30d19589c6c8226f0a734ba04"
server_key_version: "encryptionkey@0"
I20260809 00:37:00.562234 10730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "e458f0ade5934a0c97c8bd789c8b92d9"
format_stamp: "Formatted at 2026-08-09 00:37:00 on dist-test-slave-zpfg"
server_key: "966d9b3fe437acea8630cf4cfa1a318e"
server_key_iv: "7fbc5ba30d19589c6c8226f0a734ba04"
server_key_version: "encryptionkey@0"
I20260809 00:37:00.563653 10730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:00.564525 10745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:00.564743 10730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:37:00.564849 10730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "e458f0ade5934a0c97c8bd789c8b92d9"
format_stamp: "Formatted at 2026-08-09 00:37:00 on dist-test-slave-zpfg"
server_key: "966d9b3fe437acea8630cf4cfa1a318e"
server_key_iv: "7fbc5ba30d19589c6c8226f0a734ba04"
server_key_version: "encryptionkey@0"
I20260809 00:37:00.564960 10730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:00.568372 10730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:00.570523 10730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:00.570681 10730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:00.574973 10730 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34857
I20260809 00:37:00.575052 10797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34857 every 8 connection(s)
I20260809 00:37:00.575428 10730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:00.576444 10798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:00.578862 10798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9: Bootstrap starting.
I20260809 00:37:00.579452 10798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:00.579741 10798 log.cc:824] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:00.580302 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 10730
I20260809 00:37:00.580469 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:37:00.580576 10798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9: No bootstrap required, opened a new log
I20260809 00:37:00.580698 21671 external_mini_cluster.cc:1428] Setting key bc47b115ce1d86c0ac1ae566d0301ba4
I20260809 00:37:00.583250 10798 raft_consensus.cc:348] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e458f0ade5934a0c97c8bd789c8b92d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34857 } }
I20260809 00:37:00.583449 10798 raft_consensus.cc:374] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:00.583478 10798 raft_consensus.cc:729] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e458f0ade5934a0c97c8bd789c8b92d9, State: Initialized, Role: FOLLOWER
I20260809 00:37:00.583612 10798 consensus_queue.cc:260] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e458f0ade5934a0c97c8bd789c8b92d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34857 } }
I20260809 00:37:00.583698 10798 raft_consensus.cc:388] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:00.583706 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:00.581319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:58735 (local address 127.0.0.1:34857)
0809 00:37:00.581499 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:00.581506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:00.581532 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:37:00.582092 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:00.582104 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:00.582132 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:00.582259 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:00.582991 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:00.582994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:00.582994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:00.582998 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:00.583037 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:00.583047 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:00.583122 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:00.583205 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:37:00.583241 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260809 00:37:00.583721 10798 raft_consensus.cc:482] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:00.583840 10798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:00.584574 10798 raft_consensus.cc:504] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e458f0ade5934a0c97c8bd789c8b92d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34857 } }
I20260809 00:37:00.584707 10798 leader_election.cc:302] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e458f0ade5934a0c97c8bd789c8b92d9; no voters:
I20260809 00:37:00.584882 10798 leader_election.cc:288] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:00.584908 10803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:00.584954 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:00.585084 10803 raft_consensus.cc:686] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [term 1 LEADER]: Becoming Leader. State: Replica: e458f0ade5934a0c97c8bd789c8b92d9, State: Running, Role: LEADER
I20260809 00:37:00.585211 10798 sys_catalog.cc:564] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:00.585248 10803 consensus_queue.cc:237] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e458f0ade5934a0c97c8bd789c8b92d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34857 } }
I20260809 00:37:00.585669 10803 sys_catalog.cc:455] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e458f0ade5934a0c97c8bd789c8b92d9. Latest consensus state: current_term: 1 leader_uuid: "e458f0ade5934a0c97c8bd789c8b92d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e458f0ade5934a0c97c8bd789c8b92d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34857 } } }
I20260809 00:37:00.585677 10804 sys_catalog.cc:455] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e458f0ade5934a0c97c8bd789c8b92d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e458f0ade5934a0c97c8bd789c8b92d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34857 } } }
I20260809 00:37:00.585848 10804 sys_catalog.cc:458] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:00.586052 10803 sys_catalog.cc:458] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:00.586505 10812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:00.587175 10812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:00.587934 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:00.586568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:50989 (local address 127.0.0.1:34857)
0809 00:37:00.586604 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:00.586606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:00.586904 (+ 298us) server_negotiation.cc:407] Connection header received
0809 00:37:00.586958 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:00.586959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:00.586965 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:00.587019 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:00.587690 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:00.587691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:00.587692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:00.587696 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:00.587711 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:00.587716 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:00.587778 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:00.587843 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 00:37:00.587857 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:37:00.588699 10812 catalog_manager.cc:1347] Generated new cluster ID: 2f820a88cb4e4964aff82bb8011285aa
I20260809 00:37:00.588749 10812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:00.598325 10812 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:00.598551 10812 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:00.611395 10812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e458f0ade5934a0c97c8bd789c8b92d9: Generated new TSK 0
I20260809 00:37:00.611632 10812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:00.617719 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 10730
2026-08-09T00:37:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:00 dist-test-slave-zpfg krb5kdc[10829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:00 dist-test-slave-zpfg krb5kdc[10829](info): set up 2 sockets
Aug 09 00:37:00 dist-test-slave-zpfg krb5kdc[10829](info): commencing operation
krb5kdc: starting...
W20260809 00:37:01.668184 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:37:01 dist-test-slave-zpfg krb5kdc[10829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:37:01.792490 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42101
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:34111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:42101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:01.874744 10842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:01.875013 10842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:01.875058 10842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:01.876310 10842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:01.876386 10842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:01.876410 10842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:01.876430 10842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:01.876461 10842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:01.878165 10842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:34111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:42101
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:01.878482 10842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:01.878723 10842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:01.883635 10847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:01.883698 10848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:01.883649 10850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:01.883678 10842 server_base.cc:1034] running on GCE node
I20260809 00:37:01.884104 10842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:01.884438 10842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:01.885612 10842 hybrid_clock.cc:648] HybridClock initialized: now 1786235821885609 us; error 44 us; skew 500 ppm
Aug 09 00:37:01 dist-test-slave-zpfg krb5kdc[10829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235821, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:01.888188 10842 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:37:01.888703 10842 webserver.cc:460] Webserver started at http://127.21.41.254:41567/ using document root <none> and password file <none>
I20260809 00:37:01.888952 10842 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:01.889017 10842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:01.889158 10842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:01.890229 10842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "c1375c36863743df8eafb7cd12609b3f"
format_stamp: "Formatted at 2026-08-09 00:37:01 on dist-test-slave-zpfg"
server_key: "a82f54a9ecc41f0a57a9faac5dfe0dfa"
server_key_iv: "d9a203cebf06cb165c74a52c42f88685"
server_key_version: "encryptionkey@0"
I20260809 00:37:01.890650 10842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "c1375c36863743df8eafb7cd12609b3f"
format_stamp: "Formatted at 2026-08-09 00:37:01 on dist-test-slave-zpfg"
server_key: "a82f54a9ecc41f0a57a9faac5dfe0dfa"
server_key_iv: "d9a203cebf06cb165c74a52c42f88685"
server_key_version: "encryptionkey@0"
I20260809 00:37:01.891987 10842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:01.892706 10857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:01.892910 10842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:01.893006 10842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "c1375c36863743df8eafb7cd12609b3f"
format_stamp: "Formatted at 2026-08-09 00:37:01 on dist-test-slave-zpfg"
server_key: "a82f54a9ecc41f0a57a9faac5dfe0dfa"
server_key_iv: "d9a203cebf06cb165c74a52c42f88685"
server_key_version: "encryptionkey@0"
I20260809 00:37:01.893083 10842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:01.906747 10842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:01.908620 10842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:01.908795 10842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:01.912976 10842 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:42101
I20260809 00:37:01.913012 10909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:42101 every 8 connection(s)
I20260809 00:37:01.913378 10842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:01.914250 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:01.916661 10910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f: Bootstrap starting.
I20260809 00:37:01.917266 10910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:01.917541 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 10842
I20260809 00:37:01.917670 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:37:01.917654 10910 log.cc:824] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:01.917900 21671 external_mini_cluster.cc:1428] Setting key 82057e83c6ee35207d83d08677d427d0
I20260809 00:37:01.918545 10910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f: No bootstrap required, opened a new log
I20260809 00:37:01.920852 10910 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1375c36863743df8eafb7cd12609b3f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42101 } }
I20260809 00:37:01.921025 10910 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:01.921074 10910 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1375c36863743df8eafb7cd12609b3f, State: Initialized, Role: FOLLOWER
I20260809 00:37:01.921242 10910 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1375c36863743df8eafb7cd12609b3f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42101 } }
I20260809 00:37:01.921329 10910 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:01.921373 10910 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:01.921422 10910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:01.922024 10910 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1375c36863743df8eafb7cd12609b3f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42101 } }
I20260809 00:37:01.922150 10910 leader_election.cc:302] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1375c36863743df8eafb7cd12609b3f; no voters:
I20260809 00:37:01.922367 10910 leader_election.cc:288] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:01.922408 10915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:01.922535 10915 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [term 1 LEADER]: Becoming Leader. State: Replica: c1375c36863743df8eafb7cd12609b3f, State: Running, Role: LEADER
I20260809 00:37:01.922654 10910 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:01.922677 10915 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1375c36863743df8eafb7cd12609b3f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42101 } }
I20260809 00:37:01.923295 10916 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1375c36863743df8eafb7cd12609b3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1375c36863743df8eafb7cd12609b3f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42101 } } }
I20260809 00:37:01.923385 10916 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:01.923692 10917 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1375c36863743df8eafb7cd12609b3f. Latest consensus state: current_term: 1 leader_uuid: "c1375c36863743df8eafb7cd12609b3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1375c36863743df8eafb7cd12609b3f" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42101 } } }
I20260809 00:37:01.923839 10917 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:01.923974 10924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:01.924597 10924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 00:37:01 dist-test-slave-zpfg krb5kdc[10829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:37:01.926865 10924 catalog_manager.cc:1347] Generated new cluster ID: 38e0ea6d48ad4bd897645fdd347a9494
I20260809 00:37:01.926923 10924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:01.929468 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:01.918475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37389 (local address 127.21.41.254:42101)
0809 00:37:01.918682 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:01.918690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:01.918720 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:37:01.919148 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:01.919154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:01.919200 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:01.919348 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:01.919999 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:01.920885 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:01.921842 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:01.922380 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:01.922497 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:01.925612 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:01.925616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:01.925617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:01.925629 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:01.927709 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:01.928171 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:01.928175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:01.928175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:01.928294 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:01.928531 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:01.928531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:01.928532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:01.928854 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:01.929040 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:01.929142 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 00:37:01.929249 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
I20260809 00:37:01.931025 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:01.931241 10924 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:01.931933 10924 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:01.937356 10924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1375c36863743df8eafb7cd12609b3f: Generated new TSK 0
I20260809 00:37:01.937538 10924 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:01.940336 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:01.932564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40001 (local address 127.21.41.254:42101)
0809 00:37:01.932601 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:01.932602 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:01.932890 (+ 288us) server_negotiation.cc:407] Connection header received
0809 00:37:01.932938 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:01.932939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:01.932950 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:01.933005 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:01.933610 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:01.934443 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:01.935461 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:01.935805 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:01.935891 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:01.937407 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:01.937408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:01.937409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:01.937416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:01.939201 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:01.939570 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:01.939571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:01.939571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:01.939633 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:01.939833 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:01.939834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:01.939834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:01.939968 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:01.940063 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:01.940178 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 00:37:01.940242 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:37:01.942492 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 10842
W20260809 00:37:01.947391 10827 connection.cc:535] client connection to 127.21.41.254:42101 recv error: Network error: failed to read from TLS socket (remote: 127.21.41.254:42101): Connection reset by peer (error 104)
2026-08-09T00:37:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:01 dist-test-slave-zpfg krb5kdc[10941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:02 dist-test-slave-zpfg krb5kdc[10941](info): set up 2 sockets
Aug 09 00:37:02 dist-test-slave-zpfg krb5kdc[10941](info): commencing operation
krb5kdc: starting...
W20260809 00:37:02.984963 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:37:03 dist-test-slave-zpfg krb5kdc[10941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:37:03.113214 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42999
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:42999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:03.198562 10954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:03.198832 10954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:03.198870 10954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:03.200814 10954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:03.200878 10954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:03.200904 10954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:03.200927 10954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:03.200954 10954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:03.203992 10954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:42999
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:42999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:03.204478 10954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:03.204811 10954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:03.211716 10960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:03.211716 10959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:03.211916 10962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:03.211939 10954 server_base.cc:1034] running on GCE node
I20260809 00:37:03.212240 10954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:03.212770 10954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:03.214002 10954 hybrid_clock.cc:648] HybridClock initialized: now 1786235823213952 us; error 39 us; skew 500 ppm
Aug 09 00:37:03 dist-test-slave-zpfg krb5kdc[10941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235823, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:03.217952 10954 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:37:03.218497 10954 webserver.cc:460] Webserver started at http://127.21.41.254:44383/ using document root <none> and password file <none>
I20260809 00:37:03.218788 10954 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:03.218868 10954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:03.219008 10954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:03.220477 10954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "04e4450e5fa94413bc94e893f0cf079a"
format_stamp: "Formatted at 2026-08-09 00:37:03 on dist-test-slave-zpfg"
server_key: "8e9fb1d1343665a76ea02ccd9efb6ab7"
server_key_iv: "45e8184c13d4bc1130e3bbec6bab978b"
server_key_version: "encryptionkey@0"
I20260809 00:37:03.220925 10954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "04e4450e5fa94413bc94e893f0cf079a"
format_stamp: "Formatted at 2026-08-09 00:37:03 on dist-test-slave-zpfg"
server_key: "8e9fb1d1343665a76ea02ccd9efb6ab7"
server_key_iv: "45e8184c13d4bc1130e3bbec6bab978b"
server_key_version: "encryptionkey@0"
I20260809 00:37:03.222584 10954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:03.223466 10969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:03.223628 10954 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:03.223716 10954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "04e4450e5fa94413bc94e893f0cf079a"
format_stamp: "Formatted at 2026-08-09 00:37:03 on dist-test-slave-zpfg"
server_key: "8e9fb1d1343665a76ea02ccd9efb6ab7"
server_key_iv: "45e8184c13d4bc1130e3bbec6bab978b"
server_key_version: "encryptionkey@0"
I20260809 00:37:03.223807 10954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:03.239691 10954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:03.242869 10954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:03.243063 10954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:03.248781 10954 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:42999
I20260809 00:37:03.248829 11021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:42999 every 8 connection(s)
I20260809 00:37:03.249284 10954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:03.250468 11022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:03.253103 11022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a: Bootstrap starting.
I20260809 00:37:03.253712 11022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:03.253968 11022 log.cc:824] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:03.254750 11022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a: No bootstrap required, opened a new log
I20260809 00:37:03.256335 11022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4450e5fa94413bc94e893f0cf079a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42999 } }
I20260809 00:37:03.256475 11022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:03.256500 11022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04e4450e5fa94413bc94e893f0cf079a, State: Initialized, Role: FOLLOWER
I20260809 00:37:03.256605 11022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4450e5fa94413bc94e893f0cf079a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42999 } }
I20260809 00:37:03.256688 11022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:03.256729 11022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:03.256779 11022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:03.257639 11022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4450e5fa94413bc94e893f0cf079a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42999 } }
I20260809 00:37:03.257786 11022 leader_election.cc:302] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04e4450e5fa94413bc94e893f0cf079a; no voters:
I20260809 00:37:03.257982 11022 leader_election.cc:288] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:03.258014 11025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:03.258195 11025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [term 1 LEADER]: Becoming Leader. State: Replica: 04e4450e5fa94413bc94e893f0cf079a, State: Running, Role: LEADER
I20260809 00:37:03.258278 11022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:03.258282 11025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4450e5fa94413bc94e893f0cf079a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42999 } }
I20260809 00:37:03.258749 11026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04e4450e5fa94413bc94e893f0cf079a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4450e5fa94413bc94e893f0cf079a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42999 } } }
I20260809 00:37:03.258852 11026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:03.258749 11027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04e4450e5fa94413bc94e893f0cf079a. Latest consensus state: current_term: 1 leader_uuid: "04e4450e5fa94413bc94e893f0cf079a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4450e5fa94413bc94e893f0cf079a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 42999 } } }
I20260809 00:37:03.259020 11027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:03.259223 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 10954
I20260809 00:37:03.259352 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:37:03.259526 21671 external_mini_cluster.cc:1428] Setting key a4b59bfb1e1c4f8d448a06e7b4d1409d
W20260809 00:37:03.260838 11043 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:37:03.260907 11043 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:37:03.260962 11034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:03.261176 11034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:03.262687 11034 catalog_manager.cc:1347] Generated new cluster ID: 6e97dd0040614e7385f14b43ad514554
I20260809 00:37:03.262739 11034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:03.265178 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:03.260047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39715 (local address 127.21.41.254:42999)
0809 00:37:03.260304 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:03.260316 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:03.260492 (+ 176us) server_negotiation.cc:407] Connection header received
0809 00:37:03.260622 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:03.260630 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:03.260694 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:03.260825 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:03.261169 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:03.262462 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:03.263328 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:03.263751 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:03.263873 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:03.264575 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:03.264579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:03.264579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:03.264759 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:03.264788 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:03.264799 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:03.264894 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:03.264974 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 00:37:03.265002 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":128,"threads_started":1}
I20260809 00:37:03.266089 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:03.270718 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:03.267501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41333 (local address 127.21.41.254:42999)
0809 00:37:03.267539 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:03.267541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:03.267662 (+ 121us) server_negotiation.cc:407] Connection header received
0809 00:37:03.267728 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:03.267729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:03.267750 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:03.267795 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:03.267934 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:03.268402 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:03.269105 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:03.269680 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:03.269762 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:03.270467 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:03.270468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:03.270469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:03.270473 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:03.270487 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:03.270491 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:03.270544 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:03.270604 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 00:37:03.270617 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:37:03.276573 11034 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:03.277354 11034 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:03.286386 11034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04e4450e5fa94413bc94e893f0cf079a: Generated new TSK 0
I20260809 00:37:03.286614 11034 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:03.292548 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 10954
W20260809 00:37:03.297307 10939 connection.cc:535] client connection to 127.21.41.254:42999 recv error: Network error: failed to read from TLS socket (remote: 127.21.41.254:42999): Connection reset by peer (error 104)
2026-08-09T00:37:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1349 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:03 dist-test-slave-zpfg krb5kdc[11053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:03 dist-test-slave-zpfg krb5kdc[11053](info): set up 2 sockets
Aug 09 00:37:03 dist-test-slave-zpfg krb5kdc[11053](info): commencing operation
krb5kdc: starting...
W20260809 00:37:04.396147 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:37:04 dist-test-slave-zpfg krb5kdc[11053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:37:04.524946 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:40371
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:36149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:40371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:04.608060 11066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:04.608266 11066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:04.608289 11066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:04.609437 11066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:04.609483 11066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:04.609498 11066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:04.609510 11066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:04.609522 11066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:04.611063 11066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:36149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:40371
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:40371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:04.611315 11066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:04.611555 11066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:04.616362 11072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:04.616343 11071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:04.616326 11074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:04.616783 11066 server_base.cc:1034] running on GCE node
I20260809 00:37:04.617045 11066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:04.617338 11066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:04.618500 11066 hybrid_clock.cc:648] HybridClock initialized: now 1786235824618497 us; error 42 us; skew 500 ppm
Aug 09 00:37:04 dist-test-slave-zpfg krb5kdc[11053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235824, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:04.626399 11066 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:37:04.626907 11066 webserver.cc:460] Webserver started at http://127.21.41.254:44597/ using document root <none> and password file <none>
I20260809 00:37:04.627158 11066 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:04.627228 11066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:04.627372 11066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:04.628492 11066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "a7ff36e5df81498c94d639c8f7db1eeb"
format_stamp: "Formatted at 2026-08-09 00:37:04 on dist-test-slave-zpfg"
server_key: "034c1730a5d83dc7b3d6d75043cf8b5d"
server_key_iv: "8e96b6bd64a50cbfd3a0b39c4da551b3"
server_key_version: "encryptionkey@0"
I20260809 00:37:04.628877 11066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "a7ff36e5df81498c94d639c8f7db1eeb"
format_stamp: "Formatted at 2026-08-09 00:37:04 on dist-test-slave-zpfg"
server_key: "034c1730a5d83dc7b3d6d75043cf8b5d"
server_key_iv: "8e96b6bd64a50cbfd3a0b39c4da551b3"
server_key_version: "encryptionkey@0"
I20260809 00:37:04.630259 11066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:04.631033 11081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:04.631197 11066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:04.631304 11066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "a7ff36e5df81498c94d639c8f7db1eeb"
format_stamp: "Formatted at 2026-08-09 00:37:04 on dist-test-slave-zpfg"
server_key: "034c1730a5d83dc7b3d6d75043cf8b5d"
server_key_iv: "8e96b6bd64a50cbfd3a0b39c4da551b3"
server_key_version: "encryptionkey@0"
I20260809 00:37:04.631377 11066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:04.634775 11066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:04.636677 11066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:04.636850 11066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:04.640951 11066 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:40371
I20260809 00:37:04.640990 11133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:40371 every 8 connection(s)
I20260809 00:37:04.641320 11066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:04.642436 11134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:04.644757 11134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb: Bootstrap starting.
I20260809 00:37:04.645292 11134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:04.645555 11134 log.cc:824] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:04.646294 11134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb: No bootstrap required, opened a new log
I20260809 00:37:04.647898 11134 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ff36e5df81498c94d639c8f7db1eeb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40371 } }
I20260809 00:37:04.648039 11134 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:04.648063 11134 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ff36e5df81498c94d639c8f7db1eeb, State: Initialized, Role: FOLLOWER
I20260809 00:37:04.648231 11134 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ff36e5df81498c94d639c8f7db1eeb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40371 } }
I20260809 00:37:04.648295 11134 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:04.648317 11134 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:04.648346 11134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:04.649235 11134 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ff36e5df81498c94d639c8f7db1eeb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40371 } }
I20260809 00:37:04.649326 11134 leader_election.cc:302] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7ff36e5df81498c94d639c8f7db1eeb; no voters:
I20260809 00:37:04.649539 11134 leader_election.cc:288] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:04.649591 11137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:04.649816 11134 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:04.649820 11137 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [term 1 LEADER]: Becoming Leader. State: Replica: a7ff36e5df81498c94d639c8f7db1eeb, State: Running, Role: LEADER
I20260809 00:37:04.649946 11137 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ff36e5df81498c94d639c8f7db1eeb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40371 } }
I20260809 00:37:04.650319 11139 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7ff36e5df81498c94d639c8f7db1eeb. Latest consensus state: current_term: 1 leader_uuid: "a7ff36e5df81498c94d639c8f7db1eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ff36e5df81498c94d639c8f7db1eeb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40371 } } }
I20260809 00:37:04.650372 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 11066
I20260809 00:37:04.650421 11139 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:04.650488 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:37:04.650599 11138 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7ff36e5df81498c94d639c8f7db1eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ff36e5df81498c94d639c8f7db1eeb" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 40371 } } }
I20260809 00:37:04.650664 11138 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:04.650731 21671 external_mini_cluster.cc:1428] Setting key 29663d1a8ff217ed99fcfd7a69e5a177
I20260809 00:37:04.650916 11146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:04.651593 11146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:04.653349 11146 catalog_manager.cc:1347] Generated new cluster ID: 8587d136f6df418ebf982b37dc2f5ec5
I20260809 00:37:04.653405 11146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:04.653671 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:04.651416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57503 (local address 127.21.41.254:40371)
0809 00:37:04.651605 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:04.651773 (+ 168us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:04.651812 (+ 39us) server_negotiation.cc:407] Connection header received
0809 00:37:04.652273 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:04.652280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:04.652306 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:04.652498 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:04.653212 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:04.653215 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:04.653216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:04.653219 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:04.653260 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:04.653271 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:04.653340 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:04.653412 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 00:37:04.653457 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260809 00:37:04.654573 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:04.659736 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:04.655970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42337 (local address 127.21.41.254:40371)
0809 00:37:04.656010 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:04.656012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:04.658666 (+ 2654us) server_negotiation.cc:407] Connection header received
0809 00:37:04.658750 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:04.658751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:04.658758 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:04.658806 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:04.659458 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:04.659459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:04.659459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:04.659464 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:04.659480 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:04.659485 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:04.659543 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:04.659619 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 00:37:04.659633 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:37:04.667321 11146 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:04.667519 11146 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:04.673604 11146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7ff36e5df81498c94d639c8f7db1eeb: Generated new TSK 0
I20260809 00:37:04.673820 11146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:04.675911 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 11066
2026-08-09T00:37:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1383 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:04 dist-test-slave-zpfg krb5kdc[11165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:04 dist-test-slave-zpfg krb5kdc[11165](info): set up 2 sockets
Aug 09 00:37:04 dist-test-slave-zpfg krb5kdc[11165](info): commencing operation
krb5kdc: starting...
W20260809 00:37:05.784626 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:37:05 dist-test-slave-zpfg krb5kdc[11165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:37:05.910874 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:41867
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:41867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:05.998545 11178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:05.998783 11178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:05.998808 11178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:06.000015 11178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:06.000061 11178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:06.000077 11178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:06.000089 11178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:06.000140 11178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:06.001719 11178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:41867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:41867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:06.001987 11178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:06.002251 11178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:37:06.008409 11178 server_base.cc:1034] running on GCE node
W20260809 00:37:06.008343 11184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:06.008325 11183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:06.008312 11186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:06.008999 11178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:06.009277 11178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:06.010447 11178 hybrid_clock.cc:648] HybridClock initialized: now 1786235826010425 us; error 29 us; skew 500 ppm
Aug 09 00:37:06 dist-test-slave-zpfg krb5kdc[11165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235826, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:06.013240 11178 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:37:06.013660 11178 webserver.cc:460] Webserver started at http://127.21.41.254:42707/ using document root <none> and password file <none>
I20260809 00:37:06.013897 11178 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:06.013942 11178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:06.014037 11178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:06.015190 11178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "ca7f159c9bfc4202a5f4b27a82f3ab77"
format_stamp: "Formatted at 2026-08-09 00:37:06 on dist-test-slave-zpfg"
server_key: "23f9283915e9ce2a84dbfa372e61d245"
server_key_iv: "b8a959f3023a17c2eee4c1336c8b9cd2"
server_key_version: "encryptionkey@0"
I20260809 00:37:06.015515 11178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "ca7f159c9bfc4202a5f4b27a82f3ab77"
format_stamp: "Formatted at 2026-08-09 00:37:06 on dist-test-slave-zpfg"
server_key: "23f9283915e9ce2a84dbfa372e61d245"
server_key_iv: "b8a959f3023a17c2eee4c1336c8b9cd2"
server_key_version: "encryptionkey@0"
I20260809 00:37:06.016949 11178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:06.017704 11193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:06.017936 11178 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:06.018005 11178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "ca7f159c9bfc4202a5f4b27a82f3ab77"
format_stamp: "Formatted at 2026-08-09 00:37:06 on dist-test-slave-zpfg"
server_key: "23f9283915e9ce2a84dbfa372e61d245"
server_key_iv: "b8a959f3023a17c2eee4c1336c8b9cd2"
server_key_version: "encryptionkey@0"
I20260809 00:37:06.018075 11178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:06.069677 11178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:06.072167 11178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:06.072288 11178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:06.076450 11178 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:41867
I20260809 00:37:06.076527 11245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:41867 every 8 connection(s)
I20260809 00:37:06.076881 11178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:06.077154 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 11178
I20260809 00:37:06.077297 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:37:06.077523 21671 external_mini_cluster.cc:1428] Setting key 09d302133fc3e400aef1d01d044bf86f
I20260809 00:37:06.077869 11246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:06.081291 11246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77: Bootstrap starting.
I20260809 00:37:06.082002 11246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:06.082374 11246 log.cc:824] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:06.083233 11246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77: No bootstrap required, opened a new log
Aug 09 00:37:06 dist-test-slave-zpfg krb5kdc[11165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:37:06.085618 11246 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f159c9bfc4202a5f4b27a82f3ab77" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41867 } }
I20260809 00:37:06.085781 11246 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:06.085834 11246 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca7f159c9bfc4202a5f4b27a82f3ab77, State: Initialized, Role: FOLLOWER
I20260809 00:37:06.085978 11246 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f159c9bfc4202a5f4b27a82f3ab77" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41867 } }
I20260809 00:37:06.086061 11246 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:06.086103 11246 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:06.086158 11246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:06.086829 11246 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f159c9bfc4202a5f4b27a82f3ab77" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41867 } }
I20260809 00:37:06.086946 11246 leader_election.cc:302] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca7f159c9bfc4202a5f4b27a82f3ab77; no voters:
I20260809 00:37:06.087121 11246 leader_election.cc:288] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:06.087181 11251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:06.087335 11251 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [term 1 LEADER]: Becoming Leader. State: Replica: ca7f159c9bfc4202a5f4b27a82f3ab77, State: Running, Role: LEADER
I20260809 00:37:06.087412 11246 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:06.087491 11251 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f159c9bfc4202a5f4b27a82f3ab77" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41867 } }
I20260809 00:37:06.088253 11253 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca7f159c9bfc4202a5f4b27a82f3ab77. Latest consensus state: current_term: 1 leader_uuid: "ca7f159c9bfc4202a5f4b27a82f3ab77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f159c9bfc4202a5f4b27a82f3ab77" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41867 } } }
I20260809 00:37:06.088339 11253 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:06.088593 11252 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca7f159c9bfc4202a5f4b27a82f3ab77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f159c9bfc4202a5f4b27a82f3ab77" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 41867 } } }
I20260809 00:37:06.088685 11252 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:06.088739 11260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:06.088963 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:06.078031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43401 (local address 127.21.41.254:41867)
0809 00:37:06.078222 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:06.078232 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:06.078266 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:37:06.078742 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:06.078749 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:06.078798 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:06.078950 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:06.079748 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:06.080685 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:06.081738 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:06.082188 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:06.082271 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:06.085030 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:06.085035 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:06.085035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:06.085047 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:06.086851 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:06.087317 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:06.087320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:06.087320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:06.087432 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:06.087705 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:06.087707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:06.087707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:06.087997 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:06.088176 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:06.088553 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 00:37:06.088717 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260809 00:37:06.089690 11260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:06.090281 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:06.091667 11260 catalog_manager.cc:1347] Generated new cluster ID: 22b004d71f5f499fbb779052a43ce768
I20260809 00:37:06.091725 11260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:06.101405 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:06.091923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35975 (local address 127.21.41.254:41867)
0809 00:37:06.091955 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:06.091957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:06.094511 (+ 2554us) server_negotiation.cc:407] Connection header received
0809 00:37:06.094608 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:06.094609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:06.094618 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:06.094645 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:06.095285 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:06.096039 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:06.097021 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:06.097311 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:06.097347 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:06.098767 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:06.098769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:06.098769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:06.098775 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:06.100310 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:06.100647 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:06.100648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:06.100649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:06.100693 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:06.100880 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:06.100881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:06.100881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:06.100989 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:06.101063 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:06.101244 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 00:37:06.101313 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 00:37:06.111455 11260 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:06.111982 11260 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:06.122670 11260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca7f159c9bfc4202a5f4b27a82f3ab77: Generated new TSK 0
I20260809 00:37:06.122896 11260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:06.127385 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 11178
2026-08-09T00:37:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:06 dist-test-slave-zpfg krb5kdc[11277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:06 dist-test-slave-zpfg krb5kdc[11277](info): set up 2 sockets
Aug 09 00:37:06 dist-test-slave-zpfg krb5kdc[11277](info): commencing operation
krb5kdc: starting...
W20260809 00:37:07.252116 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:37:07 dist-test-slave-zpfg krb5kdc[11277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:37:07.373646 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:33149
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:40647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:33149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:07.457167 11290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:07.457386 11290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:07.457409 11290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:07.458564 11290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:07.458604 11290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:07.458618 11290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:07.458631 11290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:07.458643 11290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:07.460100 11290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:40647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:33149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:33149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:07.460302 11290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:07.460489 11290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:07.465469 11298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:07.465494 11296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:07.465469 11295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:07.465852 11290 server_base.cc:1034] running on GCE node
I20260809 00:37:07.466099 11290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:07.466464 11290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:07.467634 11290 hybrid_clock.cc:648] HybridClock initialized: now 1786235827467623 us; error 33 us; skew 500 ppm
Aug 09 00:37:07 dist-test-slave-zpfg krb5kdc[11277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235827, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:07.470252 11290 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:37:07.470767 11290 webserver.cc:460] Webserver started at http://127.21.41.254:35613/ using document root <none> and password file <none>
I20260809 00:37:07.471017 11290 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:07.471086 11290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:07.471231 11290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:07.472311 11290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "a7c7ca4354bd4356a690277e93b2a680"
format_stamp: "Formatted at 2026-08-09 00:37:07 on dist-test-slave-zpfg"
server_key: "220cf24667357c0c2fa2a6bf44052c10"
server_key_iv: "db595b57be782f69de656a96311dbf86"
server_key_version: "encryptionkey@0"
I20260809 00:37:07.472671 11290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "a7c7ca4354bd4356a690277e93b2a680"
format_stamp: "Formatted at 2026-08-09 00:37:07 on dist-test-slave-zpfg"
server_key: "220cf24667357c0c2fa2a6bf44052c10"
server_key_iv: "db595b57be782f69de656a96311dbf86"
server_key_version: "encryptionkey@0"
I20260809 00:37:07.474040 11290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:37:07.474875 11305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:07.475051 11290 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:07.475172 11290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "a7c7ca4354bd4356a690277e93b2a680"
format_stamp: "Formatted at 2026-08-09 00:37:07 on dist-test-slave-zpfg"
server_key: "220cf24667357c0c2fa2a6bf44052c10"
server_key_iv: "db595b57be782f69de656a96311dbf86"
server_key_version: "encryptionkey@0"
I20260809 00:37:07.475250 11290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:07.509403 11290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:07.511857 11290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:07.512032 11290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:07.516243 11290 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:33149
I20260809 00:37:07.516281 11357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:33149 every 8 connection(s)
I20260809 00:37:07.516664 11290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:07.517568 11358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:07.519117 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 11290
I20260809 00:37:07.519243 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:37:07.519436 21671 external_mini_cluster.cc:1428] Setting key 0826d86c4d1f562605888c956e2f063a
I20260809 00:37:07.520417 11358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680: Bootstrap starting.
I20260809 00:37:07.521112 11358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:07.521416 11358 log.cc:824] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:07.522145 11358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680: No bootstrap required, opened a new log
I20260809 00:37:07.524677 11358 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c7ca4354bd4356a690277e93b2a680" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33149 } }
I20260809 00:37:07.524835 11358 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:07.524879 11358 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7c7ca4354bd4356a690277e93b2a680, State: Initialized, Role: FOLLOWER
I20260809 00:37:07.525066 11358 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c7ca4354bd4356a690277e93b2a680" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33149 } }
I20260809 00:37:07.525177 11358 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:07.525233 11358 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:07.525271 11358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:07.525418 11361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:07.519999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54879 (local address 127.21.41.254:33149)
0809 00:37:07.520174 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:07.520182 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:07.520207 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:37:07.520650 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:07.520658 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:07.520707 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:07.520826 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:07.521090 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:07.522276 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:07.523392 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:07.523794 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:07.523877 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:07.524641 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:07.524644 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:07.524645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:07.524811 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:07.524840 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:07.524851 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:07.524933 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:07.525012 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 00:37:07.525090 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260809 00:37:07.525997 11358 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c7ca4354bd4356a690277e93b2a680" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33149 } }
I20260809 00:37:07.526126 11358 leader_election.cc:302] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7c7ca4354bd4356a690277e93b2a680; no voters:
I20260809 00:37:07.526325 11358 leader_election.cc:288] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:07.526376 11363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:07.526393 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:07.526569 11363 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [term 1 LEADER]: Becoming Leader. State: Replica: a7c7ca4354bd4356a690277e93b2a680, State: Running, Role: LEADER
I20260809 00:37:07.526619 11358 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:07.526741 11363 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c7ca4354bd4356a690277e93b2a680" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33149 } }
I20260809 00:37:07.527217 11364 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7c7ca4354bd4356a690277e93b2a680" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c7ca4354bd4356a690277e93b2a680" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33149 } } }
I20260809 00:37:07.527310 11364 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:07.528108 11372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:07.528419 11365 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7c7ca4354bd4356a690277e93b2a680. Latest consensus state: current_term: 1 leader_uuid: "a7c7ca4354bd4356a690277e93b2a680" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c7ca4354bd4356a690277e93b2a680" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 33149 } } }
I20260809 00:37:07.528533 11365 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:07.528858 11372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:07.530586 11372 catalog_manager.cc:1347] Generated new cluster ID: 470f8e9d5f02442780086f2c6545f2dc
I20260809 00:37:07.530668 11372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:07.531860 11361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:07.528432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45375 (local address 127.21.41.254:33149)
0809 00:37:07.528487 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:07.528489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:07.528694 (+ 205us) server_negotiation.cc:407] Connection header received
0809 00:37:07.528795 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:07.528796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:07.528807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:07.528835 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:07.529051 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:07.529753 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:07.530655 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:07.530872 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:07.530953 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:07.531618 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:07.531619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:07.531619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:07.531624 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:07.531638 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:07.531642 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:07.531697 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:07.531752 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 00:37:07.531775 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 00:37:07.538847 11372 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:07.539559 11372 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:07.545101 11372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7c7ca4354bd4356a690277e93b2a680: Generated new TSK 0
I20260809 00:37:07.545303 11372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:07.552945 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 11290
2026-08-09T00:37:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:07 dist-test-slave-zpfg krb5kdc[11389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:07 dist-test-slave-zpfg krb5kdc[11389](info): set up 2 sockets
Aug 09 00:37:07 dist-test-slave-zpfg krb5kdc[11389](info): commencing operation
krb5kdc: starting...
W20260809 00:37:08.628224 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:37:08 dist-test-slave-zpfg krb5kdc[11389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:37:08.751362 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:39675
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:39675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:08.838733 11402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:08.838953 11402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:08.838975 11402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:08.840106 11402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:08.840148 11402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:08.840162 11402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:08.840175 11402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:08.840188 11402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:08.841629 11402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:39675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:39675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:08.841827 11402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:08.842001 11402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:08.847110 11408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:08.847141 11407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:08.847391 11410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:08.847246 11402 server_base.cc:1034] running on GCE node
I20260809 00:37:08.847640 11402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:08.847949 11402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:08.849117 11402 hybrid_clock.cc:648] HybridClock initialized: now 1786235828849105 us; error 45 us; skew 500 ppm
Aug 09 00:37:08 dist-test-slave-zpfg krb5kdc[11389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235828, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:08.851641 11402 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:37:08.852079 11402 webserver.cc:460] Webserver started at http://127.21.41.254:35517/ using document root <none> and password file <none>
I20260809 00:37:08.852275 11402 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:08.852316 11402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:08.852415 11402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:08.853449 11402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "3307cb78bcd441aaa4d1039efda23fdf"
format_stamp: "Formatted at 2026-08-09 00:37:08 on dist-test-slave-zpfg"
server_key: "1f45c7774240eb8f9182df91655aee53"
server_key_iv: "0cad558e34eddf4356530b5b45aff959"
server_key_version: "encryptionkey@0"
I20260809 00:37:08.853752 11402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "3307cb78bcd441aaa4d1039efda23fdf"
format_stamp: "Formatted at 2026-08-09 00:37:08 on dist-test-slave-zpfg"
server_key: "1f45c7774240eb8f9182df91655aee53"
server_key_iv: "0cad558e34eddf4356530b5b45aff959"
server_key_version: "encryptionkey@0"
I20260809 00:37:08.855142 11402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:37:08.855837 11417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:08.856014 11402 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:08.856103 11402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "3307cb78bcd441aaa4d1039efda23fdf"
format_stamp: "Formatted at 2026-08-09 00:37:08 on dist-test-slave-zpfg"
server_key: "1f45c7774240eb8f9182df91655aee53"
server_key_iv: "0cad558e34eddf4356530b5b45aff959"
server_key_version: "encryptionkey@0"
I20260809 00:37:08.856164 11402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:08.859548 11402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:08.861618 11402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:08.861776 11402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:08.865695 11402 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:39675
I20260809 00:37:08.865762 11469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:39675 every 8 connection(s)
I20260809 00:37:08.866119 11402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:08.866473 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 11402
I20260809 00:37:08.866602 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:37:08.866849 21671 external_mini_cluster.cc:1428] Setting key 356fed5d686ac1a5bba8f5bb4f70c479
I20260809 00:37:08.867463 11470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:08.869545 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:08.867474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37567 (local address 127.21.41.254:39675)
0809 00:37:08.867661 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:08.867670 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:08.867700 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:37:08.868235 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:08.868244 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:08.868278 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:08.868432 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:08.869103 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:08.869107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:08.869107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:08.869111 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:08.869159 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:08.869172 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:08.869249 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:08.869286 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 00:37:08.869329 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260809 00:37:08.870766 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:08.871037 11470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf: Bootstrap starting.
I20260809 00:37:08.871829 11470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:08.872221 11470 log.cc:824] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:08.873070 11470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf: No bootstrap required, opened a new log
I20260809 00:37:08.873601 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:08.872496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52665 (local address 127.21.41.254:39675)
0809 00:37:08.872537 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:08.872539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:08.872614 (+ 75us) server_negotiation.cc:407] Connection header received
0809 00:37:08.872694 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:08.872695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:08.872700 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:08.872773 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:08.873388 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:08.873389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:08.873389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:37:08.873394 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:08.873411 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:37:08.873415 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:08.873473 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:08.873520 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 00:37:08.873533 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:37:08.875254 11470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3307cb78bcd441aaa4d1039efda23fdf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39675 } }
I20260809 00:37:08.875396 11470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:37:08.875420 11470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3307cb78bcd441aaa4d1039efda23fdf, State: Initialized, Role: FOLLOWER
I20260809 00:37:08.875567 11470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3307cb78bcd441aaa4d1039efda23fdf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39675 } }
I20260809 00:37:08.875660 11470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:37:08.875715 11470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:37:08.875767 11470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:08.876561 11470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3307cb78bcd441aaa4d1039efda23fdf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39675 } }
I20260809 00:37:08.876690 11470 leader_election.cc:302] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3307cb78bcd441aaa4d1039efda23fdf; no voters:
I20260809 00:37:08.876897 11470 leader_election.cc:288] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:37:08.876965 11480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:08.877135 11480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [term 1 LEADER]: Becoming Leader. State: Replica: 3307cb78bcd441aaa4d1039efda23fdf, State: Running, Role: LEADER
I20260809 00:37:08.877199 11470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:08.877295 11480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3307cb78bcd441aaa4d1039efda23fdf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39675 } }
I20260809 00:37:08.877691 11481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3307cb78bcd441aaa4d1039efda23fdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3307cb78bcd441aaa4d1039efda23fdf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39675 } } }
I20260809 00:37:08.877767 11482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3307cb78bcd441aaa4d1039efda23fdf. Latest consensus state: current_term: 1 leader_uuid: "3307cb78bcd441aaa4d1039efda23fdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3307cb78bcd441aaa4d1039efda23fdf" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39675 } } }
I20260809 00:37:08.877786 11481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:08.877820 11482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:08.878018 11488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:08.878880 11488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:08.880352 11488 catalog_manager.cc:1347] Generated new cluster ID: 67a325a393e345e8ac8feca666ae9b5a
I20260809 00:37:08.880393 11488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:08.887316 11488 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:08.887485 11488 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:08.896629 11488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3307cb78bcd441aaa4d1039efda23fdf: Generated new TSK 0
I20260809 00:37:08.896883 11488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:08.899505 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 11402
2026-08-09T00:37:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1346 ms)
[----------] 18 tests from AuthTokenIssuingTest (24577 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:08 dist-test-slave-zpfg krb5kdc[11501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:08 dist-test-slave-zpfg krb5kdc[11501](info): set up 2 sockets
Aug 09 00:37:08 dist-test-slave-zpfg krb5kdc[11501](info): commencing operation
krb5kdc: starting...
W20260809 00:37:09.992323 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:37:10.116691 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45695
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:45695,127.21.41.253:33163,127.21.41.252:39555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:10.199234 11514 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:37:10.199465 11514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:10.199486 11514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:10.199509 11514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:10.200897 11514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:10.200944 11514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:10.200961 11514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:10.200975 11514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:10.200989 11514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:10.202582 11514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:45695,127.21.41.253:33163,127.21.41.252:39555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:45695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:10.202792 11514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:10.202976 11514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:10.208274 11520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:10.208299 11522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:10.208318 11519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:10.208544 11514 server_base.cc:1034] running on GCE node
I20260809 00:37:10.208766 11514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:10.209015 11514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:10.210180 11514 hybrid_clock.cc:648] HybridClock initialized: now 1786235830210167 us; error 39 us; skew 500 ppm
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:10.217374 11514 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:37:10.217913 11514 webserver.cc:460] Webserver started at http://127.21.41.254:40189/ using document root <none> and password file <none>
I20260809 00:37:10.218153 11514 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:10.218222 11514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:10.218430 11514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:10.219579 11514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "19e0d951c87c4d3db696aac71bc7dee7"
format_stamp: "Formatted at 2026-08-09 00:37:10 on dist-test-slave-zpfg"
server_key: "30bff98ed8c527543e1b3d7a06610ce4"
server_key_iv: "89e6b44693cd2a3ddfb496e974a8e125"
server_key_version: "encryptionkey@0"
I20260809 00:37:10.219971 11514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "19e0d951c87c4d3db696aac71bc7dee7"
format_stamp: "Formatted at 2026-08-09 00:37:10 on dist-test-slave-zpfg"
server_key: "30bff98ed8c527543e1b3d7a06610ce4"
server_key_iv: "89e6b44693cd2a3ddfb496e974a8e125"
server_key_version: "encryptionkey@0"
I20260809 00:37:10.221486 11514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:10.222276 11529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:10.222599 11514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:10.222704 11514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "19e0d951c87c4d3db696aac71bc7dee7"
format_stamp: "Formatted at 2026-08-09 00:37:10 on dist-test-slave-zpfg"
server_key: "30bff98ed8c527543e1b3d7a06610ce4"
server_key_iv: "89e6b44693cd2a3ddfb496e974a8e125"
server_key_version: "encryptionkey@0"
I20260809 00:37:10.222786 11514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:10.244244 11514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:10.246299 11514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:10.246505 11514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:10.250680 11514 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:45695
I20260809 00:37:10.250722 11581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:45695 every 8 connection(s)
I20260809 00:37:10.251109 11514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:10.252164 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:10.252240 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 11514
I20260809 00:37:10.252370 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:37:10.252571 21671 external_mini_cluster.cc:1428] Setting key 1a95d3a4f2ef0d7e143117502c4b26ce
I20260809 00:37:10.254071 11582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.41.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.253@KRBTEST.COM" created.
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:37:10.267089 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.254900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45695 (local address 127.0.0.1:56806)
0809 00:37:10.255110 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.255190 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:10.255763 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:10.256078 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:10.256081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:10.256390 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:10.256561 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.256566 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.257728 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.257729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.258525 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.258528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.259111 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.259112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.259192 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.259973 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:10.259984 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:10.263404 (+ 3420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:10.265475 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.265482 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.265484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.265677 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.265927 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.265927 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.265928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.266038 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.266414 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:10.266420 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:10.266482 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 00:37:10.266588 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 00:37:10.266826 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":91,"thread_start_us":98,"threads_started":1}
I20260809 00:37:10.267091 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.254940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56806 (local address 127.21.41.254:45695)
0809 00:37:10.255143 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.255163 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.255305 (+ 142us) server_negotiation.cc:407] Connection header received
0809 00:37:10.255939 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.255942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.255969 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.256071 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.256678 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.257594 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.258655 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.258986 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.259133 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.263516 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.263524 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.263524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.263531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.265372 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.265779 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.265782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.265783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.265861 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.266105 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.266106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.266106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.266322 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.266439 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.266621 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.266817 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":106,"threads_started":1}
I20260809 00:37:10.267927 11582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:10.268296 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.268175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:33163 (local address 127.0.0.1:44056)
0809 00:37:10.268219 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.268269 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:33163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 00:37:10.268469 11533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.41.253:33163: connect: Connection refused (error 111)
I20260809 00:37:10.268918 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.268832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:33163 (local address 127.0.0.1:44070)
0809 00:37:10.268875 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.268901 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:33163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 00:37:10.269038 11582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.253:33163: Network error: Client connection negotiation failed: client connection to 127.21.41.253:33163: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab.
Entry for principal kudu/127.21.41.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab.
I20260809 00:37:10.279233 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.253:33163
--webserver_interface=127.21.41.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab
--principal=kudu/127.21.41.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:45695,127.21.41.253:33163,127.21.41.252:39555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:37:10.318260 11582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } attempt: 1
I20260809 00:37:10.318887 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.318717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:33163 (local address 127.0.0.1:44074)
0809 00:37:10.318791 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.318845 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:33163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:37:10.319265 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.319173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:33163 (local address 127.0.0.1:44084)
0809 00:37:10.319218 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.319246 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:33163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 00:37:10.319373 11582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.253:33163: Network error: Client connection negotiation failed: client connection to 127.21.41.253:33163: connect: Connection refused (error 111)
W20260809 00:37:10.363130 11589 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:37:10.363340 11589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:10.363358 11589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:10.363409 11589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:10.364605 11589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:10.364647 11589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:10.364661 11589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:10.364696 11589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:10.364722 11589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:10.366227 11589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:45695,127.21.41.253:33163,127.21.41.252:39555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab
--principal=kudu/127.21.41.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.253:33163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.41.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:10.366514 11589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:10.366750 11589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:10.371732 11595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:10.371742 11594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:10.371742 11597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:10.371870 11589 server_base.cc:1034] running on GCE node
I20260809 00:37:10.372159 11589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:10.372486 11589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:10.373648 11589 hybrid_clock.cc:648] HybridClock initialized: now 1786235830373637 us; error 42 us; skew 500 ppm
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:10.376278 11589 init.cc:377] Logged in from keytab as kudu/127.21.41.253@KRBTEST.COM (short username kudu)
I20260809 00:37:10.376780 11589 webserver.cc:460] Webserver started at http://127.21.41.253:38091/ using document root <none> and password file <none>
I20260809 00:37:10.377048 11589 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:10.377121 11589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:10.377279 11589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:10.378616 11589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/data/instance:
uuid: "b2e2531193c14fbdad05f9f307db6040"
format_stamp: "Formatted at 2026-08-09 00:37:10 on dist-test-slave-zpfg"
server_key: "863ba662b7f3b8d85a6e4b0caf13864c"
server_key_iv: "7fc46bf82a14b9fbc10b3dd415dcc395"
server_key_version: "encryptionkey@0"
I20260809 00:37:10.379014 11589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/wal/instance:
uuid: "b2e2531193c14fbdad05f9f307db6040"
format_stamp: "Formatted at 2026-08-09 00:37:10 on dist-test-slave-zpfg"
server_key: "863ba662b7f3b8d85a6e4b0caf13864c"
server_key_iv: "7fc46bf82a14b9fbc10b3dd415dcc395"
server_key_version: "encryptionkey@0"
I20260809 00:37:10.380502 11589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:37:10.381227 11604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:10.381419 11589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:10.381526 11589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/wal
uuid: "b2e2531193c14fbdad05f9f307db6040"
format_stamp: "Formatted at 2026-08-09 00:37:10 on dist-test-slave-zpfg"
server_key: "863ba662b7f3b8d85a6e4b0caf13864c"
server_key_iv: "7fc46bf82a14b9fbc10b3dd415dcc395"
server_key_version: "encryptionkey@0"
I20260809 00:37:10.381597 11589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:10.387539 11582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } attempt: 2
I20260809 00:37:10.387971 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.387843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:33163 (local address 127.0.0.1:44092)
0809 00:37:10.387899 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.387939 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:33163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 00:37:10.388296 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.388218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:33163 (local address 127.0.0.1:44100)
0809 00:37:10.388255 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.388280 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:33163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 00:37:10.388396 11582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.253:33163: Network error: Client connection negotiation failed: client connection to 127.21.41.253:33163: connect: Connection refused (error 111)
I20260809 00:37:10.397925 11589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:10.399979 11589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:10.400134 11589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:10.404407 11589 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.253:33163
I20260809 00:37:10.404456 11656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.253:33163 every 8 connection(s)
I20260809 00:37:10.404812 11589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
I20260809 00:37:10.405823 11657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:10.407477 11657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:37:10.414909 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 11589
I20260809 00:37:10.415015 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-1/wal/instance
I20260809 00:37:10.415226 21671 external_mini_cluster.cc:1428] Setting key ac118c489dd992f2704461268539ac66
I20260809 00:37:10.416456 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.408163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56808 (local address 127.21.41.254:45695)
0809 00:37:10.408229 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.408231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.408452 (+ 221us) server_negotiation.cc:407] Connection header received
0809 00:37:10.408912 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.408913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.408923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.408977 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.409555 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.410058 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.411058 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.411261 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.411325 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.413503 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.413504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.413505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.413515 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.414967 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.415460 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.415461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.415462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.415528 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.415765 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.415766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.415766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.415919 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.415986 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.416221 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.416314 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:37:10.416486 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.408147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45695 (local address 127.0.0.1:56808)
0809 00:37:10.408340 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.408375 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:10.408807 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:10.409032 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:10.409036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:10.409276 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:10.409476 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.409480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.410137 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.410138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.410984 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.410987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.411352 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.411353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.411457 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.411945 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:10.411953 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:10.413407 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:10.415092 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.415099 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.415101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.415389 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.415597 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.415598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.415598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.415706 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.415988 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:10.415993 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:10.416106 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 00:37:10.416185 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:37:10.416306 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
I20260809 00:37:10.417079 11657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.41.252@KRBTEST.COM; defaulting to no policy
I20260809 00:37:10.425472 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.417359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:33163 (local address 127.0.0.1:44112)
0809 00:37:10.417411 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.417423 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:10.417468 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:10.417775 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:10.417776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:10.417919 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:10.417964 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.417965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.419300 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.419301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.419886 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.419888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.420616 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.420617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.420671 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.421113 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:10.421117 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:10.422585 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:10.424379 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.424380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.424380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.424555 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.424758 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.424759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.424759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.424820 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.425100 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:10.425100 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:10.425161 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 00:37:10.425234 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:37:10.425362 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:37:10.425596 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.417445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.21.41.253:33163)
0809 00:37:10.417593 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.417598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.417620 (+ 22us) server_negotiation.cc:407] Connection header received
0809 00:37:10.417638 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.417640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.417652 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.417762 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.418051 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.419205 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.419983 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.420533 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.420622 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.422670 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.422678 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.422678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.422684 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.424302 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.424625 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.424627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.424628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.424677 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.424878 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.424879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.424879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.425036 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.425104 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.425270 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.425464 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260809 00:37:10.426177 11657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:10.426512 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.426407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56252)
0809 00:37:10.426450 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.426493 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.252@KRBTEST.COM" created.
W20260809 00:37:10.426644 11605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
I20260809 00:37:10.427057 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.426966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56254)
0809 00:37:10.427004 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.427032 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 00:37:10.427170 11657 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:39555: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab.
Entry for principal kudu/127.21.41.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab.
I20260809 00:37:10.440980 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.252:39555
--webserver_interface=127.21.41.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab
--principal=kudu/127.21.41.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:45695,127.21.41.253:33163,127.21.41.252:39555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:37:10.476431 11657 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } attempt: 1
I20260809 00:37:10.477039 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.476864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56260)
0809 00:37:10.476938 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.476995 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 00:37:10.477470 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.477342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56262)
0809 00:37:10.477401 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.477436 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 00:37:10.477604 11657 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:39555: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
I20260809 00:37:10.479537 11582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } attempt: 3
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
I20260809 00:37:10.487002 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.479850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.21.41.253:33163)
0809 00:37:10.479909 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.479911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.479938 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:37:10.480026 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.480027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.480039 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.480103 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.480428 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.480919 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.481609 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.481840 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.481914 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.484017 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.484019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.484019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.484027 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.485714 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.486095 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.486097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.486098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.486179 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.486425 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.486426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.486427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.486554 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.486635 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.486814 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.486897 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:37:10.487006 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.479795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:33163 (local address 127.0.0.1:44114)
0809 00:37:10.479882 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.479896 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:10.479957 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:10.480109 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:10.480110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:10.480279 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:10.480348 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.480350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.481005 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.481006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.481525 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.481527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.481912 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.481913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.481993 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.482536 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:10.482540 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:10.483929 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:10.485805 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.485807 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.485807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.486004 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.486263 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.486263 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.486264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.486332 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.486628 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:10.486629 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:10.486697 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 00:37:10.486786 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 00:37:10.486897 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 00:37:10.487417 11582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:10.487790 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.487690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56268)
0809 00:37:10.487738 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.487769 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 00:37:10.488106 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.488015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56280)
0809 00:37:10.488058 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.488084 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:37:10.488214 11582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:39555: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
I20260809 00:37:10.519358 11582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } attempt: 1
I20260809 00:37:10.519869 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.519717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56284)
0809 00:37:10.519792 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.519838 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 00:37:10.520267 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.520173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56300)
0809 00:37:10.520220 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.520248 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 00:37:10.520392 11582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:39555: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
W20260809 00:37:10.524930 11664 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:37:10.525149 11664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:10.525185 11664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:10.525226 11664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:10.526468 11664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:10.526523 11664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:10.526551 11664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:10.526577 11664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:10.526604 11664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:10.528129 11664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:45695,127.21.41.253:33163,127.21.41.252:39555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab
--principal=kudu/127.21.41.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.252:39555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.41.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:10.528404 11664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:10.528640 11664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:10.533591 11672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:10.533584 11669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:10.533584 11670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:10.533885 11664 server_base.cc:1034] running on GCE node
I20260809 00:37:10.534138 11664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:10.534498 11664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:10.535668 11664 hybrid_clock.cc:648] HybridClock initialized: now 1786235830535661 us; error 36 us; skew 500 ppm
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:10.538213 11664 init.cc:377] Logged in from keytab as kudu/127.21.41.252@KRBTEST.COM (short username kudu)
I20260809 00:37:10.538717 11664 webserver.cc:460] Webserver started at http://127.21.41.252:32777/ using document root <none> and password file <none>
I20260809 00:37:10.538914 11664 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:10.538955 11664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:10.539044 11664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:10.540057 11664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/data/instance:
uuid: "680879d5979743c6814974f01ceb64b4"
format_stamp: "Formatted at 2026-08-09 00:37:10 on dist-test-slave-zpfg"
server_key: "eb2f6ef137e34e708e1f5bdd76025acd"
server_key_iv: "d5e609ec438ad6f81119464ce041f18e"
server_key_version: "encryptionkey@0"
I20260809 00:37:10.540366 11664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/wal/instance:
uuid: "680879d5979743c6814974f01ceb64b4"
format_stamp: "Formatted at 2026-08-09 00:37:10 on dist-test-slave-zpfg"
server_key: "eb2f6ef137e34e708e1f5bdd76025acd"
server_key_iv: "d5e609ec438ad6f81119464ce041f18e"
server_key_version: "encryptionkey@0"
I20260809 00:37:10.541628 11664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:37:10.542387 11679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:10.542558 11664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:10.542644 11664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/wal
uuid: "680879d5979743c6814974f01ceb64b4"
format_stamp: "Formatted at 2026-08-09 00:37:10 on dist-test-slave-zpfg"
server_key: "eb2f6ef137e34e708e1f5bdd76025acd"
server_key_iv: "d5e609ec438ad6f81119464ce041f18e"
server_key_version: "encryptionkey@0"
I20260809 00:37:10.542721 11664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:10.545794 11657 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } attempt: 2
I20260809 00:37:10.546260 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.546092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56302)
0809 00:37:10.546146 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.546229 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 00:37:10.546629 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.546547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56308)
0809 00:37:10.546586 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.546612 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 00:37:10.546730 11657 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:39555: Network error: Client connection negotiation failed: client connection to 127.21.41.252:39555: connect: Connection refused (error 111)
I20260809 00:37:10.569362 11664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:10.571437 11664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:10.571557 11664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:10.575664 11664 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.252:39555
I20260809 00:37:10.575696 11731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.252:39555 every 8 connection(s)
I20260809 00:37:10.576082 11664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
I20260809 00:37:10.576226 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 11664
I20260809 00:37:10.576339 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786235559853195-21671-0/minicluster-data/master-2/wal/instance
I20260809 00:37:10.576561 21671 external_mini_cluster.cc:1428] Setting key c10544db1dc9645aa43571f75c2870e7
I20260809 00:37:10.577371 11732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:10.579561 11732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:37:10.587726 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.577259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56310 (local address 127.21.41.252:39555)
0809 00:37:10.577472 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.577481 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.577515 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:37:10.578105 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.578115 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.578182 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.578370 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.579014 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.580256 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.581382 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.581796 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.581914 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.584462 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.584471 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.584472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.584479 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.586167 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.586581 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.586584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.586584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.586677 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.586885 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.586886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.586886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.587136 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.587296 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.587387 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.587514 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":103,"threads_started":1}
I20260809 00:37:10.588014 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.580359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56818 (local address 127.21.41.254:45695)
0809 00:37:10.580441 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.580443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.580592 (+ 149us) server_negotiation.cc:407] Connection header received
0809 00:37:10.580666 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.580667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.580677 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.580731 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.581275 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.581720 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.582726 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.582921 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.583001 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.585404 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.585405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.585406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.585414 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.586814 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.587238 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.587240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.587240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.587303 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.587494 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.587494 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.587495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.587601 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.587672 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.587844 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.587924 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:37:10.588033 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.580284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:45695 (local address 127.0.0.1:56818)
0809 00:37:10.580483 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.580527 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:10.580609 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:10.580769 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:10.580771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:10.581098 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:10.581178 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.581180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.581866 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.581868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.582627 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.582630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.583035 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.583036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.583098 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.583518 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:10.583528 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:10.585299 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:10.586928 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.586935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.586937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.587150 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.587367 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.587368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.587369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.587432 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.587665 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:10.587672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:10.587720 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 00:37:10.587817 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 00:37:10.587924 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":127,"threads_started":1}
I20260809 00:37:10.588634 11732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:37:10.595475 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.589015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44124 (local address 127.21.41.253:33163)
0809 00:37:10.589068 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.589070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.589079 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:10.589106 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.589107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.589117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.589175 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.589471 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.589958 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.590728 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.590993 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.591068 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.592813 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.592814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.592815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.592821 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.594413 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.594727 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.594728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.594729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.594791 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.594968 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.594969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.594970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.595092 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.595141 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.595314 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.595395 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:37:10.595472 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.588945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:33163 (local address 127.0.0.1:44124)
0809 00:37:10.588983 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.588991 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:10.589022 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:10.589193 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:10.589193 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:10.589345 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:10.589396 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.589397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.590076 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.590076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.590647 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.590648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.591071 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.591072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.591123 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.591603 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:10.591608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:10.592733 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:10.594490 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.594491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.594491 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.594655 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.594861 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.594862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.594862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.594911 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.595153 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:10.595154 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:10.595202 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 00:37:10.595276 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:37:10.595355 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 00:37:10.595539 11582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } attempt: 2
I20260809 00:37:10.595885 11732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:10.596592 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.588629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.21.41.254:45695)
0809 00:37:10.588664 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.588666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.588673 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:37:10.588691 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.588692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.588700 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.588765 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.589323 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.589837 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.590594 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.590875 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.590926 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.593467 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.593468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.593468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.593477 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.594919 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.595287 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.595288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.595288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.595345 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.595602 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.595603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.595603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.595776 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.595962 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.596177 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.596492 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
I20260809 00:37:10.603143 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.595793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56318)
0809 00:37:10.595880 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.595890 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:10.595924 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:10.596136 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:10.596137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:10.596356 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:10.596397 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.596399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.597169 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.597170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.597783 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.597785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.598178 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.598179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.598241 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.598655 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:10.598660 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:10.599989 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:10.602007 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.602008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.602009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.602183 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.602438 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.602439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.602439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.602508 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.602815 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:10.602816 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:10.602867 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 00:37:10.602916 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 00:37:10.603018 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 00:37:10.603520 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.595992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56318 (local address 127.21.41.252:39555)
0809 00:37:10.596026 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.596027 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.596033 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:37:10.596051 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.596052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.596060 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.596117 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.596476 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.597102 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.597885 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.598097 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.598177 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.600060 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.600061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.600061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.600068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.601875 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.602255 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.602256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.602257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.602314 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.602573 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.602574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.602574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.602718 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.603342 (+ 624us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.603386 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.603464 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:37:10.605809 11582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7: Bootstrap starting.
I20260809 00:37:10.605906 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.597251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44128 (local address 127.21.41.253:33163)
0809 00:37:10.597284 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.597286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.597292 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:37:10.597365 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.597366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.597374 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.597420 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.598069 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.598727 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.599339 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.599543 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.599616 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.602958 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.602960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.602960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.602967 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.604618 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.605155 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.605156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.605157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.605209 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.605421 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.605422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.605422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.605530 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.605692 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.605728 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.605804 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:37:10.606691 11582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:10.607003 11582 log.cc:824] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:10.607173 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.596183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56322)
0809 00:37:10.596270 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.596281 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:10.596318 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:10.596924 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:10.596924 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:10.597069 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:10.597118 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.597120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.599084 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.599085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.599869 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.599872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.600314 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.600314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.600356 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.600684 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:10.600691 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:10.602653 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:10.605967 (+ 3314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.605968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.605969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.606164 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.606428 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.606429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.606429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.606485 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.606822 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:10.606831 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:10.606871 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 00:37:10.606944 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:37:10.607061 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 00:37:10.607380 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:10.607436 11737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.596220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56322 (local address 127.21.41.252:39555)
0809 00:37:10.596783 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.596784 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.596792 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:37:10.596819 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.596819 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.596828 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.596890 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.597195 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.598556 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.599958 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.600241 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.600317 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.602739 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.602740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.602740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.602747 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.605793 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.606250 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.606251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.606251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.606315 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.606561 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.606562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.606562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.606734 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.606798 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.606992 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.607125 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":538,"thread_start_us":116,"threads_started":1}
I20260809 00:37:10.608949 11732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4: Bootstrap starting.
I20260809 00:37:10.608349 11556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "19e0d951c87c4d3db696aac71bc7dee7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56826
I20260809 00:37:10.609644 11732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:10.609931 11732 log.cc:824] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:10.610569 11582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7: No bootstrap required, opened a new log
I20260809 00:37:10.610743 11732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4: No bootstrap required, opened a new log
I20260809 00:37:10.611588 11582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } }
I20260809 00:37:10.611726 11582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e0d951c87c4d3db696aac71bc7dee7, State: Initialized, Role: FOLLOWER
I20260809 00:37:10.611819 11582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } }
I20260809 00:37:10.612088 11740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } } }
I20260809 00:37:10.612138 11582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:10.612170 11740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:10.612246 11556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 00:37:10.612320 11556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:10.612768 11732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } }
I20260809 00:37:10.612936 11732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 680879d5979743c6814974f01ceb64b4, State: Initialized, Role: FOLLOWER
I20260809 00:37:10.613016 11732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } }
I20260809 00:37:10.613169 11556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } }
I20260809 00:37:10.613359 11732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:10.613302 11748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } } }
I20260809 00:37:10.613410 11748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:10.613943 11706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19e0d951c87c4d3db696aac71bc7dee7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "680879d5979743c6814974f01ceb64b4"
I20260809 00:37:10.614102 11706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [term 0 FOLLOWER]: Advancing to term 1
W20260809 00:37:10.614519 11752 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:37:10.614593 11752 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:37:10.615486 11706 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19e0d951c87c4d3db696aac71bc7dee7 in term 1.
I20260809 00:37:10.616003 11530 leader_election.cc:302] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e0d951c87c4d3db696aac71bc7dee7, 680879d5979743c6814974f01ceb64b4; no voters:
I20260809 00:37:10.616138 11740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:10.616281 11740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [term 1 LEADER]: Becoming Leader. State: Replica: 19e0d951c87c4d3db696aac71bc7dee7, State: Running, Role: LEADER
I20260809 00:37:10.616406 11740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } }
I20260809 00:37:10.616938 11765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19e0d951c87c4d3db696aac71bc7dee7. Latest consensus state: current_term: 1 leader_uuid: "19e0d951c87c4d3db696aac71bc7dee7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } } }
I20260809 00:37:10.617235 11765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:10.614730 11514 master_runner.cc:186] Error getting master registration for 127.21.41.253:33163: OK, instance_id { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" instance_seqno: 1786235830403224 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:37:10.617595 11514 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:37:10.613442 11556 leader_election.cc:288] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [CANDIDATE]: Term 1 election: Requested vote from peers b2e2531193c14fbdad05f9f307db6040 (127.21.41.253:33163), 680879d5979743c6814974f01ceb64b4 (127.21.41.252:39555)
I20260809 00:37:10.617515 11767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:10.619213 11767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:10.621044 11706 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [term 1 FOLLOWER]: Refusing update from remote peer 19e0d951c87c4d3db696aac71bc7dee7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:10.621423 11765 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:10.619434 11664 master_runner.cc:186] Error getting master registration for 127.21.41.253:33163: OK, instance_id { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" instance_seqno: 1786235830403224 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:37:10.621734 11664 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:37:10.623085 11748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19e0d951c87c4d3db696aac71bc7dee7. Latest consensus state: current_term: 1 leader_uuid: "19e0d951c87c4d3db696aac71bc7dee7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } } }
I20260809 00:37:10.623189 11748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:10.627233 11765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "19e0d951c87c4d3db696aac71bc7dee7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } } }
I20260809 00:37:10.627337 11765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:10.627815 11748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "19e0d951c87c4d3db696aac71bc7dee7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } } }
I20260809 00:37:10.627903 11748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:10.628000 11767 catalog_manager.cc:1347] Generated new cluster ID: 53a447eceeac4266af7361a41034a47b
I20260809 00:37:10.628049 11767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:10.637862 11657 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } attempt: 3
I20260809 00:37:10.639464 11764 catalog_manager.cc:1259] Loaded cluster ID: 53a447eceeac4266af7361a41034a47b
I20260809 00:37:10.639524 11764 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4: loading cluster ID for follower catalog manager: success
W20260809 00:37:10.639609 11764 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260809 00:37:10.639639 11764 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
Aug 09 00:37:10 dist-test-slave-zpfg krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235830, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
I20260809 00:37:10.650000 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.638401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.21.41.252:39555)
0809 00:37:10.638453 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.638455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.638466 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:37:10.638511 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.638512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.638521 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.638567 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.639028 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.639772 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.641447 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.641733 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.641790 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.644928 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.644929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.644930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.644936 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.646867 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.647540 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.647542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.647542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.648787 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.649002 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.649003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.649004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.649141 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.649202 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.649826 (+ 624us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.649897 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:37:10.649580 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.638181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:39555 (local address 127.0.0.1:56344)
0809 00:37:10.638293 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:10.638304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:10.638441 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:10.638668 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:10.638669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:10.638873 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:10.638929 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.638931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.639955 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.639956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.641362 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:10.641365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.642370 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.642371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:10.642431 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.642940 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:10.642946 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:10.644835 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:10.646961 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.646962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.646963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.647456 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.648859 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:10.648860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:10.648861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:10.648928 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:10.649229 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:10.649229 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:10.649272 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 00:37:10.649350 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:37:10.649447 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 00:37:10.651933 11657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040: Bootstrap starting.
I20260809 00:37:10.652587 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.627411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56840 (local address 127.21.41.254:45695)
0809 00:37:10.627450 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.627452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.642629 (+ 15177us) server_negotiation.cc:407] Connection header received
0809 00:37:10.642722 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.642723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.642734 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.642763 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.645211 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.645910 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.647625 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.647895 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.647940 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.649387 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.649389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.649389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.649396 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.651319 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.651721 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.651722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.651723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.651780 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.651981 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.651982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.651982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.652110 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.652207 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.652420 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.652498 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:37:10.652734 11657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:10.653069 11657 log.cc:824] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:10.653563 11767 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:10.654376 11767 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:10.654966 11737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.627452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.21.41.252:39555)
0809 00:37:10.627560 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.627562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.639045 (+ 11483us) server_negotiation.cc:407] Connection header received
0809 00:37:10.639196 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.639197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.639223 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.639252 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.640957 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.641669 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.643602 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.643940 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.643984 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.646881 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.646882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.646882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.646889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.648668 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.654059 (+ 5391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.654060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.654060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.654115 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.654326 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.654327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.654327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.654490 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.654604 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.654786 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.654875 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 00:37:10.655550 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:10.638627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44140 (local address 127.21.41.253:33163)
0809 00:37:10.638678 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.638680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.641671 (+ 2991us) server_negotiation.cc:407] Connection header received
0809 00:37:10.641769 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.641770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.641781 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.641811 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.643503 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.644211 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.646910 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:10.647249 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:10.647294 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:10.649572 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:10.649573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:10.649573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:10.649580 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:10.652021 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.653103 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.653104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.653105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.653174 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:10.654323 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:10.654324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:10.654324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:10.654471 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:10.654543 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:10.655389 (+ 846us) server_negotiation.cc:299] Negotiation successful
0809 00:37:10.655467 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:37:10.656723 11657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040: No bootstrap required, opened a new log
I20260809 00:37:10.658484 11767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7: Generated new TSK 0
I20260809 00:37:10.658674 11767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:10.659366 11657 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } }
I20260809 00:37:10.659547 11657 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e2531193c14fbdad05f9f307db6040, State: Initialized, Role: FOLLOWER
I20260809 00:37:10.659655 11657 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } }
I20260809 00:37:10.659997 11657 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:10.659960 11782 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } } }
I20260809 00:37:10.660059 11631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19e0d951c87c4d3db696aac71bc7dee7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b2e2531193c14fbdad05f9f307db6040"
I20260809 00:37:10.660178 11631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:10.661038 11631 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19e0d951c87c4d3db696aac71bc7dee7 in term 1.
I20260809 00:37:10.661268 11630 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [term 1 FOLLOWER]: Refusing update from remote peer 19e0d951c87c4d3db696aac71bc7dee7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:10.660140 11782 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:10.661895 11740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:37:10.662659 11793 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:37:10.662715 11793 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:37:10.664373 11797 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19e0d951c87c4d3db696aac71bc7dee7. Latest consensus state: current_term: 1 leader_uuid: "19e0d951c87c4d3db696aac71bc7dee7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } } }
I20260809 00:37:10.664443 11797 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:10.664475 11797 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "19e0d951c87c4d3db696aac71bc7dee7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } } }
I20260809 00:37:10.664507 11797 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:37:10.689262 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.687852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44156 (local address 127.21.41.253:33163)
0809 00:37:10.687900 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.687903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.688573 (+ 670us) server_negotiation.cc:407] Connection header received
0809 00:37:10.688684 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.688685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.688697 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.688729 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.689176 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44156: BlockingRecv error: recv got EOF from 127.0.0.1:44156 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 00:37:10.690016 11586 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.688270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.21.41.254:45695)
0809 00:37:10.688309 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.688311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.689494 (+ 1183us) server_negotiation.cc:407] Connection header received
0809 00:37:10.689590 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.689591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.689602 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.689632 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.689939 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56854: BlockingRecv error: recv got EOF from 127.0.0.1:56854 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 00:37:10.691044 11737 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.690381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.21.41.252:39555)
0809 00:37:10.690435 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.690437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.690543 (+ 106us) server_negotiation.cc:407] Connection header received
0809 00:37:10.690648 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.690649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.690659 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.690688 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.690985 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56354: BlockingRecv error: recv got EOF from 127.0.0.1:56354 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 00:37:10.695462 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.694661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.21.41.253:33163)
0809 00:37:10.694698 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.694699 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.694894 (+ 195us) server_negotiation.cc:407] Connection header received
0809 00:37:10.695004 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.695005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.695024 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.695053 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.695410 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44164: BlockingRecv error: recv got EOF from 127.0.0.1:44164 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:37:10.698649 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.697737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44180 (local address 127.21.41.253:33163)
0809 00:37:10.697773 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.697775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.698039 (+ 264us) server_negotiation.cc:407] Connection header received
0809 00:37:10.698169 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.698170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.698181 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.698219 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.698590 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44180: BlockingRecv error: recv got EOF from 127.0.0.1:44180 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 00:37:10.702593 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.701739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44192 (local address 127.21.41.253:33163)
0809 00:37:10.701774 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.701776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.702034 (+ 258us) server_negotiation.cc:407] Connection header received
0809 00:37:10.702130 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.702131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.702142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.702170 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.702542 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44192: BlockingRecv error: recv got EOF from 127.0.0.1:44192 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 00:37:10.708434 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.707638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.21.41.253:33163)
0809 00:37:10.707673 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.707675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.707924 (+ 249us) server_negotiation.cc:407] Connection header received
0809 00:37:10.708019 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.708020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.708030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.708059 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.708385 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44196: BlockingRecv error: recv got EOF from 127.0.0.1:44196 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:37:10.718461 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.717541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.21.41.253:33163)
0809 00:37:10.717580 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.717582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.717839 (+ 257us) server_negotiation.cc:407] Connection header received
0809 00:37:10.717956 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.717957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.717967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.717998 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.718404 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44212: BlockingRecv error: recv got EOF from 127.0.0.1:44212 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:37:10.736630 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.735673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44226 (local address 127.21.41.253:33163)
0809 00:37:10.735720 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.735722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.736020 (+ 298us) server_negotiation.cc:407] Connection header received
0809 00:37:10.736131 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.736132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.736143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.736174 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.736569 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44226: BlockingRecv error: recv got EOF from 127.0.0.1:44226 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 00:37:10.771263 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.770124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.21.41.253:33163)
0809 00:37:10.770178 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.770180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.770545 (+ 365us) server_negotiation.cc:407] Connection header received
0809 00:37:10.770695 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.770696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.770711 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.770746 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.771196 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44232: BlockingRecv error: recv got EOF from 127.0.0.1:44232 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 00:37:10.837517 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.836767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44238 (local address 127.21.41.253:33163)
0809 00:37:10.836828 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.836830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.836846 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:37:10.836937 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.836938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.836951 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.837043 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.837439 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44238: BlockingRecv error: recv got EOF from 127.0.0.1:44238 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 00:37:10.967828 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:10.966919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44242 (local address 127.21.41.253:33163)
0809 00:37:10.967028 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:10.967031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:10.967149 (+ 118us) server_negotiation.cc:407] Connection header received
0809 00:37:10.967227 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:10.967228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:10.967241 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:10.967319 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:10.967700 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44242: BlockingRecv error: recv got EOF from 127.0.0.1:44242 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 00:37:11.147878 11740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "19e0d951c87c4d3db696aac71bc7dee7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e0d951c87c4d3db696aac71bc7dee7" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 45695 } } peers { permanent_uuid: "b2e2531193c14fbdad05f9f307db6040" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 33163 } } peers { permanent_uuid: "680879d5979743c6814974f01ceb64b4" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 39555 } } }
I20260809 00:37:11.148065 11740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19e0d951c87c4d3db696aac71bc7dee7 [sys.catalog]: This master's current role is: LEADER
W20260809 00:37:11.225885 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:11.225001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44246 (local address 127.21.41.253:33163)
0809 00:37:11.225075 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:11.225077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:11.225187 (+ 110us) server_negotiation.cc:407] Connection header received
0809 00:37:11.225239 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:11.225240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:11.225252 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:11.225309 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:11.225822 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44246: BlockingRecv error: recv got EOF from 127.0.0.1:44246 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:37:11.640794 11764 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4: acquiring CA information for follower catalog manager: success
I20260809 00:37:11.641119 11764 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 680879d5979743c6814974f01ceb64b4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:37:11.663143 11793 catalog_manager.cc:1259] Loaded cluster ID: 53a447eceeac4266af7361a41034a47b
I20260809 00:37:11.663224 11793 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040: loading cluster ID for follower catalog manager: success
I20260809 00:37:11.664453 11793 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040: acquiring CA information for follower catalog manager: success
I20260809 00:37:11.664978 11793 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b2e2531193c14fbdad05f9f307db6040: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:37:11.742079 11865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:11.739238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44256 (local address 127.21.41.253:33163)
0809 00:37:11.739373 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:11.739375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:11.739411 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:37:11.739516 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:11.739517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:11.739580 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:11.739670 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:11.739871 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:11.740429 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:11.741205 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:11.741415 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:11.741492 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:11.741687 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:11.741828 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:11.741925 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:11.741949 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 00:37:11.741972 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260809 00:37:11.746901 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:11.744180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56360 (local address 127.21.41.252:39555)
0809 00:37:11.744304 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:11.744306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:11.744315 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:11.744350 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:11.744351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:11.744388 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:11.744459 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:11.744556 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:11.745125 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:11.745953 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:11.746139 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:11.746193 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:11.746451 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:11.746591 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:11.746694 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:11.746759 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 00:37:11.746784 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":51,"threads_started":1}
I20260809 00:37:11.751761 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:11.748929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.21.41.252:39555)
0809 00:37:11.748976 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:11.748978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:11.748987 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:11.749011 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:11.749012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:11.749024 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:11.749080 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:11.749254 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:11.749910 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:11.750935 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:11.751185 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:11.751254 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:11.751405 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:11.751436 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:11.751509 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:11.751661 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 00:37:11.751686 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:37:11.752007 11865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:11.748762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44258 (local address 127.21.41.253:33163)
0809 00:37:11.748895 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:11.748897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:11.749127 (+ 230us) server_negotiation.cc:407] Connection header received
0809 00:37:11.749190 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:11.749191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:11.749202 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:11.749248 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:11.749418 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:11.750058 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:11.751166 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:11.751509 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:11.751574 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:11.751735 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:11.751766 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:11.751795 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:11.751907 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 00:37:11.751933 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260809 00:37:11.752504 11878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:11.748742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56868 (local address 127.21.41.254:45695)
0809 00:37:11.749047 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:11.749049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:11.749146 (+ 97us) server_negotiation.cc:407] Connection header received
0809 00:37:11.749272 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:11.749272 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:11.749331 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:11.749407 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:11.749617 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:11.750407 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:11.751386 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:11.751737 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:11.751802 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:11.752001 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:11.752150 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:11.752213 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:11.752384 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 00:37:11.752412 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":103,"threads_started":1}
I20260809 00:37:11.754405 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 11514
I20260809 00:37:11.760383 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 11589
I20260809 00:37:11.765491 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 11664
2026-08-09T00:37:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2867 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:37:11 dist-test-slave-zpfg krb5kdc[11883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:37:11 dist-test-slave-zpfg krb5kdc[11883](info): set up 2 sockets
Aug 09 00:37:11 dist-test-slave-zpfg krb5kdc[11883](info): commencing operation
krb5kdc: starting...
W20260809 00:37:12.900246 21671 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/test-user.keytab.
Aug 09 00:37:12 dist-test-slave-zpfg krb5kdc[11883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:37:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:37:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.41.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
Entry for principal kudu/127.21.41.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab.
I20260809 00:37:13.027446 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:39917
--webserver_interface=127.21.41.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:39917,127.21.41.253:44473,127.21.41.252:35267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:37:13.113286 11896 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:37:13.113505 11896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:13.113523 11896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:13.113543 11896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:13.114727 11896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:13.114770 11896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:13.114784 11896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:13.114797 11896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:13.114809 11896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:13.116274 11896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:39917,127.21.41.253:44473,127.21.41.252:35267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.254.keytab
--principal=kudu/127.21.41.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.254:39917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.41.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:13.116473 11896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:13.116647 11896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:13.121709 11901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:13.121820 11896 server_base.cc:1034] running on GCE node
W20260809 00:37:13.121840 11902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:13.121735 11904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:13.122227 11896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:13.122556 11896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:13.123759 11896 hybrid_clock.cc:648] HybridClock initialized: now 1786235833123736 us; error 47 us; skew 500 ppm
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:13.131943 11896 init.cc:377] Logged in from keytab as kudu/127.21.41.254@KRBTEST.COM (short username kudu)
I20260809 00:37:13.132469 11896 webserver.cc:460] Webserver started at http://127.21.41.254:36689/ using document root <none> and password file <none>
I20260809 00:37:13.132699 11896 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:13.132766 11896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:13.132920 11896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:13.134019 11896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/data/instance:
uuid: "2f259ea660f14b5c9f985bd6dc721a0a"
format_stamp: "Formatted at 2026-08-09 00:37:13 on dist-test-slave-zpfg"
server_key: "071c5bf7de4c181553200341baa26afd"
server_key_iv: "5602e787c8862cb5890144e001d713e1"
server_key_version: "encryptionkey@0"
I20260809 00:37:13.134487 11896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/wal/instance:
uuid: "2f259ea660f14b5c9f985bd6dc721a0a"
format_stamp: "Formatted at 2026-08-09 00:37:13 on dist-test-slave-zpfg"
server_key: "071c5bf7de4c181553200341baa26afd"
server_key_iv: "5602e787c8862cb5890144e001d713e1"
server_key_version: "encryptionkey@0"
I20260809 00:37:13.135901 11896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:37:13.136650 11911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:13.136852 11896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:13.136956 11896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/wal
uuid: "2f259ea660f14b5c9f985bd6dc721a0a"
format_stamp: "Formatted at 2026-08-09 00:37:13 on dist-test-slave-zpfg"
server_key: "071c5bf7de4c181553200341baa26afd"
server_key_iv: "5602e787c8862cb5890144e001d713e1"
server_key_version: "encryptionkey@0"
I20260809 00:37:13.137043 11896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:13.152897 11896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:13.154922 11896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:13.155085 11896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:13.159298 11896 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.254:39917
I20260809 00:37:13.159358 11963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.254:39917 every 8 connection(s)
I20260809 00:37:13.159751 11896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/data/info.pb
I20260809 00:37:13.160805 11964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:13.162439 11964 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:13.163867 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 11896
I20260809 00:37:13.164004 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-0/wal/instance
I20260809 00:37:13.164301 21671 external_mini_cluster.cc:1428] Setting key 2d3671ddf466323f790a296b908840d7
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.41.253@KRBTEST.COM; defaulting to no policy
I20260809 00:37:13.174968 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.163237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.21.41.254:39917)
0809 00:37:13.163456 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.163465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.163621 (+ 156us) server_negotiation.cc:407] Connection header received
0809 00:37:13.164268 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.164271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.164304 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.164413 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.165000 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.166502 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.167640 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.167966 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.168056 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.170719 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.170728 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.170729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.170738 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.173114 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.173544 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.173548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.173548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.173666 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.173936 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.173937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.173938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.174382 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.174480 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.174681 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.174839 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":177,"spinlock_wait_cycles":9600,"thread_start_us":122,"threads_started":1}
I20260809 00:37:13.174968 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.163222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:39917 (local address 127.0.0.1:49414)
0809 00:37:13.163446 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.163490 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:13.164036 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:13.164449 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:13.164453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:13.164709 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:13.164881 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.164886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.166597 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.166598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.167532 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.167534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.168087 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.168088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.168174 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.168717 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:13.168728 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:13.170622 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:13.173221 (+ 2599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.173228 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.173230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.173442 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.173745 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.173746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.173746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.173863 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.174471 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:13.174478 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:13.174542 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 00:37:13.174627 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 00:37:13.174765 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":120,"threads_started":1}
I20260809 00:37:13.175977 11964 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:13.176548 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.176359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:44473 (local address 127.0.0.1:55572)
0809 00:37:13.176436 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.176484 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 00:37:13.176710 11915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.41.253:44473: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.253@KRBTEST.COM" created.
I20260809 00:37:13.177175 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.177076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:44473 (local address 127.0.0.1:55588)
0809 00:37:13.177124 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.177157 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 00:37:13.177337 11964 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.253:44473: Network error: Client connection negotiation failed: client connection to 127.21.41.253:44473: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab.
Entry for principal kudu/127.21.41.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab.
I20260809 00:37:13.192606 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.253:44473
--webserver_interface=127.21.41.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab
--principal=kudu/127.21.41.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:39917,127.21.41.253:44473,127.21.41.252:35267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:37:13.226562 11964 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } attempt: 1
I20260809 00:37:13.227170 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.227004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:44473 (local address 127.0.0.1:55598)
0809 00:37:13.227076 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.227127 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 00:37:13.227577 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.227477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:44473 (local address 127.0.0.1:55608)
0809 00:37:13.227526 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.227555 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 00:37:13.227758 11964 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.253:44473: Network error: Client connection negotiation failed: client connection to 127.21.41.253:44473: connect: Connection refused (error 111)
W20260809 00:37:13.290457 11971 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:37:13.290776 11971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:13.290814 11971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:13.290866 11971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:13.292980 11971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:13.293049 11971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:13.293076 11971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:13.293110 11971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:13.293141 11971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:13.295960 11964 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } attempt: 2
I20260809 00:37:13.296160 11971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:39917,127.21.41.253:44473,127.21.41.252:35267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.253.keytab
--principal=kudu/127.21.41.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.253:44473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.41.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:13.296470 11971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:13.296519 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.296355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:44473 (local address 127.0.0.1:55624)
0809 00:37:13.296422 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.296480 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 00:37:13.296769 11971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:37:13.296893 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.296778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:44473 (local address 127.0.0.1:55628)
0809 00:37:13.296830 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.296859 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.253:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 00:37:13.296996 11964 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.253:44473: Network error: Client connection negotiation failed: client connection to 127.21.41.253:44473: connect: Connection refused (error 111)
W20260809 00:37:13.302345 11977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:13.302371 11976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:13.302368 11979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:13.302682 11971 server_base.cc:1034] running on GCE node
I20260809 00:37:13.302994 11971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:13.303408 11971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:13.304598 11971 hybrid_clock.cc:648] HybridClock initialized: now 1786235833304584 us; error 31 us; skew 500 ppm
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:13.307410 11971 init.cc:377] Logged in from keytab as kudu/127.21.41.253@KRBTEST.COM (short username kudu)
I20260809 00:37:13.307868 11971 webserver.cc:460] Webserver started at http://127.21.41.253:37935/ using document root <none> and password file <none>
I20260809 00:37:13.308178 11971 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:13.308246 11971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:13.308418 11971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:13.309535 11971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/data/instance:
uuid: "2c078be61b9e453ea3e9408ac5b235ab"
format_stamp: "Formatted at 2026-08-09 00:37:13 on dist-test-slave-zpfg"
server_key: "703735dcbc45be9f878b01116b1038bb"
server_key_iv: "56e3515ef862b72d2c3bb3a2fdcb29bd"
server_key_version: "encryptionkey@0"
I20260809 00:37:13.309926 11971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/wal/instance:
uuid: "2c078be61b9e453ea3e9408ac5b235ab"
format_stamp: "Formatted at 2026-08-09 00:37:13 on dist-test-slave-zpfg"
server_key: "703735dcbc45be9f878b01116b1038bb"
server_key_iv: "56e3515ef862b72d2c3bb3a2fdcb29bd"
server_key_version: "encryptionkey@0"
I20260809 00:37:13.311414 11971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 00:37:13.312252 11986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:13.312446 11971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:37:13.312562 11971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/wal
uuid: "2c078be61b9e453ea3e9408ac5b235ab"
format_stamp: "Formatted at 2026-08-09 00:37:13 on dist-test-slave-zpfg"
server_key: "703735dcbc45be9f878b01116b1038bb"
server_key_iv: "56e3515ef862b72d2c3bb3a2fdcb29bd"
server_key_version: "encryptionkey@0"
I20260809 00:37:13.312639 11971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:13.329049 11971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:13.331123 11971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:13.331300 11971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:13.335309 11971 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.253:44473
I20260809 00:37:13.335343 12038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.253:44473 every 8 connection(s)
I20260809 00:37:13.335727 11971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/data/info.pb
I20260809 00:37:13.336786 12039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:13.338565 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 11971
I20260809 00:37:13.338651 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-1/wal/instance
I20260809 00:37:13.338455 12039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:13.338867 21671 external_mini_cluster.cc:1428] Setting key 5a1d1ff6966f94b5ada12b3b413a1291
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.41.252@KRBTEST.COM; defaulting to no policy
I20260809 00:37:13.348904 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.339221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49428 (local address 127.21.41.254:39917)
0809 00:37:13.339275 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.339277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.339506 (+ 229us) server_negotiation.cc:407] Connection header received
0809 00:37:13.339967 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.339969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.339984 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.340081 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.340785 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.341565 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.342664 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.343036 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.343122 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.345665 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.345667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.345667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.345677 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.347447 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.347901 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.347903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.347903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.347958 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.348185 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.348186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.348186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.348303 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.348356 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.348712 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.348836 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:37:13.349030 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.339191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:39917 (local address 127.0.0.1:49428)
0809 00:37:13.339362 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.339404 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:13.339878 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:13.340158 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:13.340161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:13.340484 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:13.340686 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.340691 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.341673 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.341674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.342574 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.342576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.343143 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.343145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.343244 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.343811 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:13.343820 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:13.345552 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:13.347555 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.347565 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.347567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.347820 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.348023 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.348024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.348024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.348130 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.348375 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:13.348380 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:13.348557 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 00:37:13.348675 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 00:37:13.348811 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260809 00:37:13.349710 12039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.41.252@KRBTEST.COM" created.
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
I20260809 00:37:13.357744 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.350084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:44473 (local address 127.0.0.1:55644)
0809 00:37:13.350152 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.350163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:13.350210 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:13.350536 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:13.350537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:13.350681 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:13.350729 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.350731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.351614 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.351615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.352194 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.352196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.352864 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.352865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.352955 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.353324 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:13.353331 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:13.354833 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:13.356679 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.356680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.356680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.356854 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.357068 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.357069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.357069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.357121 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.357417 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:13.357418 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:13.357458 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 00:37:13.357524 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 00:37:13.357627 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 00:37:13.357786 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.350156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55644 (local address 127.21.41.253:44473)
0809 00:37:13.350382 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.350387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.350412 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:37:13.350432 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.350434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.350453 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.350536 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.350812 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.351488 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.352277 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.352778 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.352852 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.354935 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.354944 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.354945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.354951 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.356577 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.356942 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.356945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.356948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.356998 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.357190 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.357190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.357191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.357338 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.357425 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.357560 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.357679 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
I20260809 00:37:13.358443 12039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:13.358814 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.358674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52280)
0809 00:37:13.358728 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.358778 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 00:37:13.358958 11990 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
I20260809 00:37:13.359393 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.359318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52296)
0809 00:37:13.359345 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.359377 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260809 00:37:13.359514 12039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:35267: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.41.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab.
Entry for principal kudu/127.21.41.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab.
I20260809 00:37:13.366650 21671 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbJF9WP/build/release/bin/kudu
/tmp/dist-test-taskbJF9WP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.252:35267
--webserver_interface=127.21.41.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab
--principal=kudu/127.21.41.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.41.212:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.41.254:39917,127.21.41.253:44473,127.21.41.252:35267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:37:13.388146 11964 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } attempt: 3
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
I20260809 00:37:13.395402 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.388566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:44473 (local address 127.0.0.1:55648)
0809 00:37:13.388638 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.388651 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:13.388729 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:13.388896 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:13.388896 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:13.389071 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:13.389138 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.389140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.389840 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.389841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.390466 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.390468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.390878 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.390879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.390967 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.391447 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:13.391452 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:13.392665 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:13.394447 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.394448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.394448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.394619 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.394837 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.394837 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.394838 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.394891 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.395154 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:13.395154 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:13.395196 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 00:37:13.395247 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:37:13.395325 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 00:37:13.395473 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.388653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55648 (local address 127.21.41.253:44473)
0809 00:37:13.388729 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.388732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.388744 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:37:13.388798 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.388799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.388811 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.388888 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.389245 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.389745 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.390554 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.390783 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.390878 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.392751 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.392752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.392753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.392761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.394327 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.394693 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.394694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.394694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.394759 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.394950 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.394951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.394951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.395092 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.395160 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.395294 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.395380 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:37:13.396076 11964 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:13.396468 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.396359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52304)
0809 00:37:13.396411 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.396445 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:37:13.396850 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.396746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52306)
0809 00:37:13.396796 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.396819 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 00:37:13.396960 11964 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:35267: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
I20260809 00:37:13.408735 12039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } attempt: 1
I20260809 00:37:13.409190 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.409062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52322)
0809 00:37:13.409120 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.409160 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:37:13.409610 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.409519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52326)
0809 00:37:13.409564 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.409591 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:37:13.409719 12039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:35267: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
I20260809 00:37:13.428099 11964 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } attempt: 1
I20260809 00:37:13.428609 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.428460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52328)
0809 00:37:13.428521 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.428560 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 00:37:13.428951 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.428856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52332)
0809 00:37:13.428902 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.428928 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:37:13.429039 11964 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:35267: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
W20260809 00:37:13.452950 12046 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:37:13.453171 12046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:37:13.453189 12046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:37:13.453209 12046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:37:13.454407 12046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:37:13.454454 12046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:37:13.454468 12046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:37:13.454481 12046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:37:13.454493 12046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:37:13.456024 12046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.41.212:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.41.254:39917,127.21.41.253:44473,127.21.41.252:35267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/krb5kdc/kudu_127.21.41.252.keytab
--principal=kudu/127.21.41.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.41.252:35267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.41.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:37:13.456316 12046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:37:13.456561 12046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:37:13.461644 12052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:13.461663 12051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:37:13.461637 12054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:37:13.461834 12046 server_base.cc:1034] running on GCE node
I20260809 00:37:13.462100 12046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:37:13.462445 12046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:37:13.463655 12046 hybrid_clock.cc:648] HybridClock initialized: now 1786235833463627 us; error 34 us; skew 500 ppm
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:37:13.466457 12046 init.cc:377] Logged in from keytab as kudu/127.21.41.252@KRBTEST.COM (short username kudu)
I20260809 00:37:13.466887 12046 webserver.cc:460] Webserver started at http://127.21.41.252:37565/ using document root <none> and password file <none>
I20260809 00:37:13.467099 12046 fs_manager.cc:359] Metadata directory not provided
I20260809 00:37:13.467139 12046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:37:13.467235 12046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:37:13.468305 12046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/data/instance:
uuid: "9cffa0d03fc84c1fb4f393eca246f88a"
format_stamp: "Formatted at 2026-08-09 00:37:13 on dist-test-slave-zpfg"
server_key: "bde8ca9874ebc59e7dd198e233c0f84f"
server_key_iv: "c44ba54390ac4e2f7e583d7f4a2eb59c"
server_key_version: "encryptionkey@0"
I20260809 00:37:13.468622 12046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/wal/instance:
uuid: "9cffa0d03fc84c1fb4f393eca246f88a"
format_stamp: "Formatted at 2026-08-09 00:37:13 on dist-test-slave-zpfg"
server_key: "bde8ca9874ebc59e7dd198e233c0f84f"
server_key_iv: "c44ba54390ac4e2f7e583d7f4a2eb59c"
server_key_version: "encryptionkey@0"
I20260809 00:37:13.469938 12046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:37:13.470695 12061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:37:13.470918 12046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:37:13.471006 12046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/data,/tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/wal
uuid: "9cffa0d03fc84c1fb4f393eca246f88a"
format_stamp: "Formatted at 2026-08-09 00:37:13 on dist-test-slave-zpfg"
server_key: "bde8ca9874ebc59e7dd198e233c0f84f"
server_key_iv: "c44ba54390ac4e2f7e583d7f4a2eb59c"
server_key_version: "encryptionkey@0"
I20260809 00:37:13.471066 12046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:37:13.477874 12039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } attempt: 2
I20260809 00:37:13.478395 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.478206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52344)
0809 00:37:13.478274 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.478319 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:37:13.478796 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.478694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52360)
0809 00:37:13.478747 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.478775 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 00:37:13.478902 12039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.41.252:35267: Network error: Client connection negotiation failed: client connection to 127.21.41.252:35267: connect: Connection refused (error 111)
I20260809 00:37:13.489336 12046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:37:13.491758 12046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:37:13.491945 12046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:37:13.496264 12046 rpc_server.cc:307] RPC server started. Bound to: 127.21.41.252:35267
I20260809 00:37:13.496300 12113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.41.252:35267 every 8 connection(s)
I20260809 00:37:13.496717 12046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/data/info.pb
I20260809 00:37:13.497767 12114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:37:13.499462 12114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:13.502182 21671 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbJF9WP/build/release/bin/kudu as pid 12046
I20260809 00:37:13.502301 21671 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbJF9WP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786235559853195-21671-0/minicluster-data/master-2/wal/instance
I20260809 00:37:13.502514 21671 external_mini_cluster.cc:1428] Setting key 97c2e0b25ec1efb457fbb2c819ead265
I20260809 00:37:13.504176 11964 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } attempt: 2
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.254@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
I20260809 00:37:13.510461 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.500259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49438 (local address 127.21.41.254:39917)
0809 00:37:13.500341 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.500343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.500620 (+ 277us) server_negotiation.cc:407] Connection header received
0809 00:37:13.501280 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.501281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.501292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.501357 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.502008 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.502631 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.503923 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.504157 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.504226 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.506853 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.506854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.506855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.506862 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.508502 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.509001 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.509003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.509003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.509066 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.509379 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.509381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.509381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.509556 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.509632 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.510217 (+ 585us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.510307 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:37:13.510615 12116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.500251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.254:39917 (local address 127.0.0.1:49438)
0809 00:37:13.500459 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.500520 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:13.501184 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:13.501413 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:13.501416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:13.501696 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:13.501923 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.501928 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.502729 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.502730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.503797 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.503799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.504256 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.504257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.504408 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.504928 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:13.504939 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:13.506763 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:13.508602 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.508610 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.508613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.508922 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.509133 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.509134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.509134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.509292 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.509655 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:13.509663 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:13.510075 (+ 412us) client_negotiation.cc:769] Sending connection context
0809 00:37:13.510173 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 00:37:13.510325 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":105,"threads_started":1}
I20260809 00:37:13.511461 12114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:13.512590 12118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.502937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52368 (local address 127.21.41.252:35267)
0809 00:37:13.503118 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.503123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.503156 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:37:13.503208 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.503211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.503232 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.503315 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.503913 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.504861 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.505820 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.506312 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.506394 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.509236 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.509242 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.509242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.509249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.511243 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.511601 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.511605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.511605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.511683 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.511941 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.511941 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.511942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.512125 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.512337 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.512396 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.512480 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":119,"threads_started":1}
I20260809 00:37:13.514182 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.504457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52380)
0809 00:37:13.504506 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.504516 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:13.504553 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:13.505115 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:13.505116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:13.505303 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:13.505344 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.505345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.506049 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.506050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.506675 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.506678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.507150 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.507151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.507217 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.507729 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:13.507734 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:13.509647 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:13.512980 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.512981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.512981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.513199 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.513524 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.513525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.513525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.513594 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.513847 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:13.513848 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:13.513896 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 00:37:13.513981 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 00:37:13.514087 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:37:13.514211 12119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.504491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52380 (local address 127.21.41.252:35267)
0809 00:37:13.504978 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.504980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.504986 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:37:13.505005 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.505005 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.505014 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.505086 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.505439 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.505955 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.506761 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.507053 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.507157 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.509734 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.509735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.509735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.509742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.512868 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.513317 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.513318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.513318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.513389 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.513666 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.513667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.513667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.513772 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.513852 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.514018 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.514089 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":89,"threads_started":1}
I20260809 00:37:13.516237 11964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a: Bootstrap starting.
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
I20260809 00:37:13.517040 11964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:13.517349 11964 log.cc:824] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a: Log is configured to *not* fsync() on all Append() calls
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.254@KRBTEST.COM
I20260809 00:37:13.518489 11964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a: No bootstrap required, opened a new log
I20260809 00:37:13.520032 12116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.511781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.253:44473 (local address 127.0.0.1:55650)
0809 00:37:13.511836 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.511848 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:13.511886 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:13.512156 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:13.512157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:13.512419 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:13.512463 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.512464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.513331 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.513332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.514176 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.514178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.514727 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.514728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.514787 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.515193 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:13.515198 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:13.516954 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:13.518889 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.518890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.518890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.519117 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.519339 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.519340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.519341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.519404 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.519708 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:13.519709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:13.519757 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 00:37:13.519835 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:37:13.519931 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:37:13.520356 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.511950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55650 (local address 127.21.41.253:44473)
0809 00:37:13.512008 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.512010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.512019 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:13.512046 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.512047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.512057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.512107 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.512558 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.513229 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.514265 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.514597 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.514651 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.517047 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.517049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.517049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.517058 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.518793 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.519196 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.519197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.519197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.519250 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.519458 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.519459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.519459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.519614 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.519678 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.519896 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.520250 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:37:13.520668 12114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:37:13.520637 11964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } }
I20260809 00:37:13.520769 11964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f259ea660f14b5c9f985bd6dc721a0a, State: Initialized, Role: FOLLOWER
I20260809 00:37:13.520865 11964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } }
I20260809 00:37:13.521227 11964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:13.522231 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.513462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49446 (local address 127.21.41.254:39917)
0809 00:37:13.513501 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.513502 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.513511 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:13.513528 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.513528 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.513538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.513592 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.514167 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.514872 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.515950 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.516280 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.516353 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.519168 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.519169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.519170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.519177 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.521011 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.521403 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.521405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.521405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.521460 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.521698 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.521699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.521699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.521832 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.522000 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.522039 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.522113 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:37:13.523445 12132 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:37:13.523509 12132 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:37:13.523654 12121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } } }
I20260809 00:37:13.523743 12121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.252@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
I20260809 00:37:13.524276 11896 master_runner.cc:186] Error getting master registration for 127.21.41.253:44473: OK, instance_id { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" instance_seqno: 1786235833334114 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:37:13.527418 11896 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.41.253@KRBTEST.COM
I20260809 00:37:13.528644 12116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.520917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52392)
0809 00:37:13.520974 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.520987 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:13.521033 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:13.521174 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:13.521175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:13.521323 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:13.521359 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.521360 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.522299 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.522300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.523068 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.523071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.523570 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.523571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.523626 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.524060 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:13.524065 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:13.525426 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:13.527492 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.527492 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.527493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.527694 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.527999 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.528000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.528000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.528067 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.528318 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:13.528319 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:13.528356 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 00:37:13.528424 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:37:13.528532 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:37:13.528919 12119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.521001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52392 (local address 127.21.41.252:35267)
0809 00:37:13.521038 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.521041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.521069 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:37:13.521088 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.521088 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.521098 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.521142 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.521452 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.522203 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.523157 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.523453 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.523521 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.525511 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.525512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.525512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.525518 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.527402 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.527780 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.527781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.527781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.527918 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.528132 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.528133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.528134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.528252 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.528327 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.528468 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.528582 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":18}
I20260809 00:37:13.530645 12114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a: Bootstrap starting.
I20260809 00:37:13.531391 12114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:13.531708 12114 log.cc:824] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:13.532359 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.522803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55662 (local address 127.21.41.253:44473)
0809 00:37:13.522839 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.522841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.522849 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:37:13.522995 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.522996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.523007 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.523054 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.523692 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.524335 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.525312 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.525602 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.525662 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.528349 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.528351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.528351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.528358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.531066 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.531479 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.531480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.531481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.531543 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.531749 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.531750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.531750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.531890 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.532072 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.532108 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.532183 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:37:13.532507 12114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a: No bootstrap required, opened a new log
I20260809 00:37:13.533533 21671 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:37:13.534595 11938 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2f259ea660f14b5c9f985bd6dc721a0a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49446
I20260809 00:37:13.534739 11938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 00:37:13.534799 11938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:13.534691 12114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } }
I20260809 00:37:13.534854 12114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cffa0d03fc84c1fb4f393eca246f88a, State: Initialized, Role: FOLLOWER
I20260809 00:37:13.534950 12114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } }
I20260809 00:37:13.535300 12114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:13.535264 12135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } } }
I20260809 00:37:13.535450 12135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:13.535858 11938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } }
I20260809 00:37:13.536114 11938 leader_election.cc:288] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [CANDIDATE]: Term 1 election: Requested vote from peers 2c078be61b9e453ea3e9408ac5b235ab (127.21.41.253:44473), 9cffa0d03fc84c1fb4f393eca246f88a (127.21.41.252:35267)
I20260809 00:37:13.536499 12088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9cffa0d03fc84c1fb4f393eca246f88a"
I20260809 00:37:13.536620 12088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:13.538128 12088 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f259ea660f14b5c9f985bd6dc721a0a in term 1.
I20260809 00:37:13.538502 11915 leader_election.cc:302] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f259ea660f14b5c9f985bd6dc721a0a, 9cffa0d03fc84c1fb4f393eca246f88a; no voters:
I20260809 00:37:13.538637 12121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:37:13.538823 12121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [term 1 LEADER]: Becoming Leader. State: Replica: 2f259ea660f14b5c9f985bd6dc721a0a, State: Running, Role: LEADER
I20260809 00:37:13.538944 12121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } }
I20260809 00:37:13.539443 12121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f259ea660f14b5c9f985bd6dc721a0a. Latest consensus state: current_term: 1 leader_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } } }
I20260809 00:37:13.539557 12121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:13.540874 12046 master_runner.cc:186] Error getting master registration for 127.21.41.253:44473: OK, instance_id { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" instance_seqno: 1786235833334114 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:37:13.543396 12046 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:37:13.545871 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.537213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55666 (local address 127.21.41.253:44473)
0809 00:37:13.537258 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.537260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.537478 (+ 218us) server_negotiation.cc:407] Connection header received
0809 00:37:13.537582 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.537583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.537594 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.537625 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.538558 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.539319 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.540396 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.541248 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.541324 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.542996 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.542997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.542998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.543006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.544646 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.545048 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.545050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.545050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.545112 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.545329 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.545330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.545331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.545453 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.545552 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.545688 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.545768 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 00:37:13.547044 12152 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:37:13.547106 12152 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:37:13.547827 12155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:37:13.548028 12155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:37:13.550030 12088 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [term 1 FOLLOWER]: Refusing update from remote peer 2f259ea660f14b5c9f985bd6dc721a0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:13.550431 12121 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:13.552199 12135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f259ea660f14b5c9f985bd6dc721a0a. Latest consensus state: current_term: 1 leader_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } } }
I20260809 00:37:13.552326 12135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:13.554639 12119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.537447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52400 (local address 127.21.41.252:35267)
0809 00:37:13.537484 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.537486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.545909 (+ 8423us) server_negotiation.cc:407] Connection header received
0809 00:37:13.545964 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.545965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.545977 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.546031 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.546791 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.547453 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.548448 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.548737 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.548809 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.551236 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.551249 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.551249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.551257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.552962 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.553407 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.553408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.553408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.553480 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.553728 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.553729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.553729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.553890 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.553979 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.554466 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.554548 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:37:13.556342 12121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } } }
I20260809 00:37:13.556442 12121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [sys.catalog]: This master's current role is: LEADER
I20260809 00:37:13.556475 12135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } } }
I20260809 00:37:13.556512 12157 mvcc.cc:205] Tried to move back new op lower bound from 7316421974216822784 to 7316421974176436224. Current Snapshot: MvccSnapshot[applied={T|T < 7316421974216822784}]
I20260809 00:37:13.556557 12135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:13.559155 12158 mvcc.cc:205] Tried to move back new op lower bound from 7316421974216822784 to 7316421974176436224. Current Snapshot: MvccSnapshot[applied={T|T < 7316421974216822784}]
I20260809 00:37:13.559789 12155 catalog_manager.cc:1347] Generated new cluster ID: 9afccb6771eb400893794ac5368bca53
I20260809 00:37:13.559859 12155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:37:13.563915 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.555158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49460 (local address 127.21.41.254:39917)
0809 00:37:13.555207 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.555209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.555218 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:13.555279 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.555280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.555306 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.555361 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.556222 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.556958 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.558046 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.558407 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.558483 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.560117 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.560118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.560119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.560125 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.561700 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.562268 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.562269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.562269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.562363 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.562620 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.562621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.562621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.562741 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.563241 (+ 500us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.563584 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.563757 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:37:13.568946 12155 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:37:13.569792 12155 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:37:13.570039 12039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } attempt: 3
Aug 09 00:37:13 dist-test-slave-zpfg krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786235833, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.41.253@KRBTEST.COM for kudu/127.21.41.252@KRBTEST.COM
I20260809 00:37:13.578231 12155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a: Generated new TSK 0
I20260809 00:37:13.578498 12155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:37:13.579504 12119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.570437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52404 (local address 127.21.41.252:35267)
0809 00:37:13.570480 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.570482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.570550 (+ 68us) server_negotiation.cc:407] Connection header received
0809 00:37:13.570634 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.570634 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.570657 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.570710 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.571109 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.571872 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.572909 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.573326 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.573413 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.575925 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:37:13.575926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:37:13.575927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:37:13.575934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:37:13.577757 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.578279 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.578280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.578281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.578512 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:37:13.578875 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:37:13.578876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:37:13.578876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:37:13.579067 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:37:13.579135 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:13.579312 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 00:37:13.579408 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:37:13.579504 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:13.570405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.41.252:35267 (local address 127.0.0.1:52404)
0809 00:37:13.570505 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 00:37:13.570516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:37:13.570564 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:37:13.570766 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:37:13.570768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:37:13.570965 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:37:13.571021 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.571024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.571988 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.571989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.572806 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:37:13.572809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:37:13.573478 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:37:13.573480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:37:13.573567 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:13.574076 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:37:13.574082 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:37:13.575804 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:37:13.577887 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.577888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.577889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.578147 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.578714 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:37:13.578715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:37:13.578715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:37:13.578797 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:37:13.579138 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:37:13.579139 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:37:13.579187 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 00:37:13.579286 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 00:37:13.579409 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 00:37:13.582572 12039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab: Bootstrap starting.
I20260809 00:37:13.583377 12039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab: Neither blocks nor log segments found. Creating new log.
I20260809 00:37:13.583694 12039 log.cc:824] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab: Log is configured to *not* fsync() on all Append() calls
I20260809 00:37:13.589150 12039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab: No bootstrap required, opened a new log
I20260809 00:37:13.591887 12039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } }
I20260809 00:37:13.592077 12039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c078be61b9e453ea3e9408ac5b235ab, State: Initialized, Role: FOLLOWER
I20260809 00:37:13.592170 12039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } }
I20260809 00:37:13.592526 12039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:37:13.594013 12012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:37:13.594399 12013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2c078be61b9e453ea3e9408ac5b235ab"
I20260809 00:37:13.595590 12012 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [term 1 FOLLOWER]: Refusing update from remote peer 2f259ea660f14b5c9f985bd6dc721a0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:37:13.597826 12153 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:37:13.599776 12182 mvcc.cc:205] Tried to move back new op lower bound from 7316421974216822784 to 7316421974176436224. Current Snapshot: MvccSnapshot[applied={T|T < 7316421974216822784}]
W20260809 00:37:13.601719 12176 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:37:13.601791 12176 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:37:13.601953 12165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f259ea660f14b5c9f985bd6dc721a0a. Latest consensus state: current_term: 1 leader_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } } }
I20260809 00:37:13.602041 12165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:13.602098 12165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } } }
I20260809 00:37:13.602149 12165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:37:13.602288 12177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } } }
I20260809 00:37:13.602329 12177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:37:13.603619 12043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.602662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55676 (local address 127.21.41.253:44473)
0809 00:37:13.602719 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.602721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.602866 (+ 145us) server_negotiation.cc:407] Connection header received
0809 00:37:13.602983 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.602984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.602997 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.603031 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.603556 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55676: BlockingRecv error: recv got EOF from 127.0.0.1:55676 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:37:13.603678 12013 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2f259ea660f14b5c9f985bd6dc721a0a for term 1 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
W20260809 00:37:13.604552 12119 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.602939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52420 (local address 127.21.41.252:35267)
0809 00:37:13.602979 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.602980 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.603987 (+ 1007us) server_negotiation.cc:407] Connection header received
0809 00:37:13.604092 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.604093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.604104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.604136 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.604493 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52420: BlockingRecv error: recv got EOF from 127.0.0.1:52420 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:37:13.611698 11967 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.610703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49470 (local address 127.21.41.254:39917)
0809 00:37:13.610755 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.610757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.610964 (+ 207us) server_negotiation.cc:407] Connection header received
0809 00:37:13.611075 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.611076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.611090 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.611126 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.611606 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49470: BlockingRecv error: recv got EOF from 127.0.0.1:49470 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 00:37:13.624212 12043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.621332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55692 (local address 127.21.41.253:44473)
0809 00:37:13.621384 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.621386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.621582 (+ 196us) server_negotiation.cc:407] Connection header received
0809 00:37:13.621679 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.621680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.621705 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.621761 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.624135 (+ 2374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55692: BlockingRecv error: recv got EOF from 127.0.0.1:55692 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 00:37:13.627357 12043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.626583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55696 (local address 127.21.41.253:44473)
0809 00:37:13.626629 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.626631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.626870 (+ 239us) server_negotiation.cc:407] Connection header received
0809 00:37:13.626900 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.626900 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.626913 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.626968 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.627289 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55696: BlockingRecv error: recv got EOF from 127.0.0.1:55696 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 00:37:13.631553 12043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.630719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55702 (local address 127.21.41.253:44473)
0809 00:37:13.630757 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.630759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.631007 (+ 248us) server_negotiation.cc:407] Connection header received
0809 00:37:13.631104 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.631105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.631115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.631144 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.631493 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55702: BlockingRecv error: recv got EOF from 127.0.0.1:55702 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:37:13.637686 12043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.636821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55708 (local address 127.21.41.253:44473)
0809 00:37:13.636860 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.636861 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.637123 (+ 262us) server_negotiation.cc:407] Connection header received
0809 00:37:13.637226 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.637227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.637240 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.637270 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.637632 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55708: BlockingRecv error: recv got EOF from 127.0.0.1:55708 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 00:37:13.648126 12043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.647134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55714 (local address 127.21.41.253:44473)
0809 00:37:13.647178 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.647180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.647480 (+ 300us) server_negotiation.cc:407] Connection header received
0809 00:37:13.647585 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.647586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.647597 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.647627 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.648048 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55714: BlockingRecv error: recv got EOF from 127.0.0.1:55714 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 00:37:13.666774 12043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.665709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.21.41.253:44473)
0809 00:37:13.665757 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.665759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.666050 (+ 291us) server_negotiation.cc:407] Connection header received
0809 00:37:13.666157 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.666158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.666170 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.666200 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.666710 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55726: BlockingRecv error: recv got EOF from 127.0.0.1:55726 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 00:37:13.701277 12043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.700275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.21.41.253:44473)
0809 00:37:13.700339 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.700341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.700642 (+ 301us) server_negotiation.cc:407] Connection header received
0809 00:37:13.700755 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.700756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.700770 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.700801 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.701212 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55728: BlockingRecv error: recv got EOF from 127.0.0.1:55728 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:37:13.767765 12043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.766936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55732 (local address 127.21.41.253:44473)
0809 00:37:13.767030 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.767032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.767168 (+ 136us) server_negotiation.cc:407] Connection header received
0809 00:37:13.767199 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.767200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.767212 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.767257 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.767590 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55732: BlockingRecv error: recv got EOF from 127.0.0.1:55732 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 00:37:13.897768 12043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:13.897061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55746 (local address 127.21.41.253:44473)
0809 00:37:13.897132 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:13.897134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:13.897232 (+ 98us) server_negotiation.cc:407] Connection header received
0809 00:37:13.897280 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:13.897281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:13.897292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:13.897360 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:13.897680 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55746: BlockingRecv error: recv got EOF from 127.0.0.1:55746 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:37:14.147538 12153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f259ea660f14b5c9f985bd6dc721a0a" member_type: VOTER last_known_addr { host: "127.21.41.254" port: 39917 } } peers { permanent_uuid: "2c078be61b9e453ea3e9408ac5b235ab" member_type: VOTER last_known_addr { host: "127.21.41.253" port: 44473 } } peers { permanent_uuid: "9cffa0d03fc84c1fb4f393eca246f88a" member_type: VOTER last_known_addr { host: "127.21.41.252" port: 35267 } } }
I20260809 00:37:14.147716 12153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f259ea660f14b5c9f985bd6dc721a0a [sys.catalog]: This master's current role is: LEADER
W20260809 00:37:14.155896 12043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:37:14.155218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.21.41.253:44473)
0809 00:37:14.155285 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:14.155287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:14.155296 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:37:14.155322 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:14.155323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:14.155334 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:14.155385 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:37:14.155803 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55748: BlockingRecv error: recv got EOF from 127.0.0.1:55748 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:37:14.547591 12152 catalog_manager.cc:1259] Loaded cluster ID: 9afccb6771eb400893794ac5368bca53
I20260809 00:37:14.547703 12152 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a: loading cluster ID for follower catalog manager: success
I20260809 00:37:14.548527 12152 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a: acquiring CA information for follower catalog manager: success
I20260809 00:37:14.548882 12152 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9cffa0d03fc84c1fb4f393eca246f88a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:37:14.602229 12176 catalog_manager.cc:1259] Loaded cluster ID: 9afccb6771eb400893794ac5368bca53
I20260809 00:37:14.602321 12176 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab: loading cluster ID for follower catalog manager: success
I20260809 00:37:14.603308 12176 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab: acquiring CA information for follower catalog manager: success
I20260809 00:37:14.603675 12176 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2c078be61b9e453ea3e9408ac5b235ab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:37:14.671808 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:14.669165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.21.41.253:44473)
0809 00:37:14.669327 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:14.669329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:14.669343 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:37:14.669389 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:14.669390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:14.669442 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:14.669522 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:14.669688 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:14.670215 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:14.671049 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:14.671268 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:14.671327 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:14.671477 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:14.671583 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:14.671644 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:14.671679 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 00:37:14.671697 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
I20260809 00:37:14.676345 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:14.673878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.21.41.252:35267)
0809 00:37:14.674019 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:14.674021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:14.674029 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:37:14.674074 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:14.674075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:14.674111 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:14.674181 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:14.674268 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:14.674822 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:14.675574 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:14.675823 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:14.675887 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:14.675999 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:14.676101 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:14.676153 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:14.676191 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 00:37:14.676213 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260809 00:37:14.681015 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:14.678372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.21.41.252:35267)
0809 00:37:14.678408 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:14.678410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:14.678578 (+ 168us) server_negotiation.cc:407] Connection header received
0809 00:37:14.678677 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:14.678679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:14.678689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:14.678742 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:14.678874 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:14.679527 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:14.680263 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:14.680541 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:14.680608 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:14.680758 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:14.680789 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:14.680847 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:14.680923 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 00:37:14.680942 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 00:37:14.681209 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:14.678041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55764 (local address 127.21.41.253:44473)
0809 00:37:14.678079 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:14.678081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:14.678234 (+ 153us) server_negotiation.cc:407] Connection header received
0809 00:37:14.678330 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:14.678332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:14.678361 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:14.678412 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:14.678577 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:14.679340 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:14.680336 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:14.680634 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:14.680696 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:14.680921 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:14.680954 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:14.680985 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:14.681098 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 00:37:14.681119 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:37:14.681725 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:37:14.678040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49482 (local address 127.21.41.254:39917)
0809 00:37:14.678146 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 00:37:14.678148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:37:14.678177 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:37:14.678288 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:37:14.678289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:37:14.678389 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:37:14.678471 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:37:14.678630 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:14.679224 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:14.680755 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:37:14.681064 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:37:14.681125 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:37:14.681376 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:37:14.681498 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:37:14.681552 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:37:14.681632 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 00:37:14.681666 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
I20260809 00:37:14.683331 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 11896
I20260809 00:37:14.689751 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 11971
I20260809 00:37:14.694871 21671 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbJF9WP/build/release/bin/kudu with pid 12046
2026-08-09T00:37:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2929 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5796 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (274849 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST